2024-09-22T00:46:52.936 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-22T00:46:52.952 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T00:46:53.033 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914593' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1478 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 24/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVucGqkrSf/q5/xLYTxKBUYe8McKkxnlpURmHbMl6RSj4t4cUlqh8UsTs6AGqz+GLdBnQwHwYd7S03wyJyBegc= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBuhwlnBFlT6NiBF5l3x/0ibeumHbhKYQHKpQttpi4ao6q6B9ef3LEtw+5y26kkALWNOnIwTAhf2dln/30dpbTU= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-22T00:46:53.033 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-22T00:46:53.034 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-22T00:46:53.034 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-22T00:46:53.036 INFO:teuthology.task.internal:Checking packages... 2024-09-22T00:46:53.054 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-22T00:46:53.054 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-22T00:46:53.054 INFO:teuthology.packaging:ref: None 2024-09-22T00:46:53.054 INFO:teuthology.packaging:tag: None 2024-09-22T00:46:53.054 INFO:teuthology.packaging:branch: squid 2024-09-22T00:46:53.054 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T00:46:53.054 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-22T00:46:53.238 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:46:53.239 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-22T00:46:53.250 INFO:teuthology.task.internal:no buildpackages task found 2024-09-22T00:46:53.250 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-22T00:46:53.258 INFO:teuthology.task.internal:Saving configuration 2024-09-22T00:46:53.272 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-22T00:46:53.278 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-22T00:46:53.304 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 00:41:17.333500', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII6CA97t7MfujTOqR8SxNOGBUJXgq/At55mcrW4OXbQf'} 2024-09-22T00:46:53.325 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 00:41:17.331525', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDumDpDKeL+x2hVRLAJ7IJ2+vEK2VXordPJvVwis3A62'} 2024-09-22T00:46:53.325 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-22T00:46:53.332 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-22T00:46:53.332 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-22T00:46:53.332 INFO:teuthology.run_tasks:Running task console_log... 2024-09-22T00:46:53.395 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f37cbd37880>, signals=[15]) 2024-09-22T00:46:53.395 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-22T00:46:53.402 INFO:teuthology.task.internal:Opening connections... 2024-09-22T00:46:53.402 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T00:46:53.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:46:53.490 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T00:46:53.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:46:53.569 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-22T00:46:53.579 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-09-22T00:46:53.584 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-09-22T00:46:53.584 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-09-22T00:46:53.631 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-22T00:46:53.631 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Ubuntu" 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="22.04" 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_CODENAME=jammy 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:ID=ubuntu 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE=debian 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-22T00:46:53.632 INFO:teuthology.orchestra.run.smithi002.stdout:UBUNTU_CODENAME=jammy 2024-09-22T00:46:53.633 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-09-22T00:46:53.654 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-09-22T00:46:53.659 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-09-22T00:46:53.659 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu" 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="22.04" 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=jammy 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-22T00:46:53.704 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-22T00:46:53.705 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-22T00:46:53.705 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-22T00:46:53.705 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=jammy 2024-09-22T00:46:53.705 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-09-22T00:46:53.728 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-22T00:46:53.743 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-22T00:46:53.764 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-22T00:46:53.764 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2024-09-22T00:46:53.767 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-09-22T00:46:53.771 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-22T00:46:53.793 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-22T00:46:53.793 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2024-09-22T00:46:53.813 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-09-22T00:46:53.820 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-22T00:46:53.871 INFO:teuthology.run_tasks:Running task kernel... 2024-09-22T00:46:53.884 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-22T00:46:53.884 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-22T00:46:53.884 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T00:46:53.884 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T00:46:53.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:46:53.889 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-09-22T00:46:53.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:46:53.890 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-09-22T00:46:53.936 INFO:teuthology.orchestra.run.smithi119.stdout:5.15.0-56-generic 2024-09-22T00:46:53.936 INFO:teuthology.task.kernel:Running kernel on smithi119: 5.15.0-56-generic 2024-09-22T00:46:53.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get clean 2024-09-22T00:46:53.937 INFO:teuthology.orchestra.run.smithi002.stdout:5.15.0-56-generic 2024-09-22T00:46:53.938 INFO:teuthology.task.kernel:Running kernel on smithi002: 5.15.0-56-generic 2024-09-22T00:46:53.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo apt-get clean 2024-09-22T00:46:54.063 DEBUG:teuthology.orchestra.run.smithi002:> sudo apt-get update 2024-09-22T00:46:54.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-09-22T00:46:54.338 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T00:46:54.338 INFO:teuthology.orchestra.run.smithi002.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-22T00:46:54.374 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T00:46:54.376 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-22T00:46:54.393 INFO:teuthology.orchestra.run.smithi002.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-22T00:46:54.432 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-22T00:46:54.750 INFO:teuthology.orchestra.run.smithi002.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-22T00:46:54.773 INFO:teuthology.orchestra.run.smithi002.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-22T00:46:54.787 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-22T00:46:54.804 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-22T00:46:54.891 INFO:teuthology.orchestra.run.smithi002.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-22T00:46:54.925 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-22T00:46:55.069 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-22T00:46:55.084 INFO:teuthology.orchestra.run.smithi002.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-22T00:46:55.131 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-22T00:46:55.229 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-22T00:46:55.244 INFO:teuthology.orchestra.run.smithi002.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-22T00:46:55.245 INFO:teuthology.orchestra.run.smithi002.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-22T00:46:55.245 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-22T00:46:55.254 INFO:teuthology.orchestra.run.smithi119.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-22T00:46:55.254 INFO:teuthology.orchestra.run.smithi119.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-22T00:46:55.256 INFO:teuthology.orchestra.run.smithi002.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-22T00:46:55.257 INFO:teuthology.orchestra.run.smithi002.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-22T00:46:55.259 INFO:teuthology.orchestra.run.smithi002.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-22T00:46:55.259 INFO:teuthology.orchestra.run.smithi002.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-22T00:46:55.261 INFO:teuthology.orchestra.run.smithi002.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-22T00:46:55.280 INFO:teuthology.orchestra.run.smithi119.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-22T00:46:55.294 INFO:teuthology.orchestra.run.smithi119.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-22T00:46:55.295 INFO:teuthology.orchestra.run.smithi119.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-22T00:46:55.297 INFO:teuthology.orchestra.run.smithi119.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-22T00:46:55.330 INFO:teuthology.orchestra.run.smithi002.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-22T00:46:55.342 INFO:teuthology.orchestra.run.smithi002.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-22T00:46:55.351 INFO:teuthology.orchestra.run.smithi119.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-22T00:46:55.352 INFO:teuthology.orchestra.run.smithi119.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-22T00:46:55.380 INFO:teuthology.orchestra.run.smithi002.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-22T00:46:55.380 INFO:teuthology.orchestra.run.smithi002.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-22T00:46:55.380 INFO:teuthology.orchestra.run.smithi002.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-22T00:46:55.384 INFO:teuthology.orchestra.run.smithi119.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-22T00:46:55.384 INFO:teuthology.orchestra.run.smithi119.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-22T00:46:55.384 INFO:teuthology.orchestra.run.smithi119.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-22T00:46:55.388 INFO:teuthology.orchestra.run.smithi002.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-22T00:46:55.388 INFO:teuthology.orchestra.run.smithi002.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-22T00:46:55.392 INFO:teuthology.orchestra.run.smithi119.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-22T00:46:55.393 INFO:teuthology.orchestra.run.smithi119.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-22T00:46:55.399 INFO:teuthology.orchestra.run.smithi119.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-22T00:46:55.401 INFO:teuthology.orchestra.run.smithi002.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-22T00:46:55.404 INFO:teuthology.orchestra.run.smithi002.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-22T00:46:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-22T00:46:55.414 INFO:teuthology.orchestra.run.smithi002.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-22T00:46:55.415 INFO:teuthology.orchestra.run.smithi119.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-22T00:46:55.415 INFO:teuthology.orchestra.run.smithi119.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-22T00:46:55.415 INFO:teuthology.orchestra.run.smithi119.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-22T00:46:55.415 INFO:teuthology.orchestra.run.smithi002.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-22T00:46:55.416 INFO:teuthology.orchestra.run.smithi119.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-22T00:46:55.418 INFO:teuthology.orchestra.run.smithi002.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-22T00:46:55.418 INFO:teuthology.orchestra.run.smithi002.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-22T00:46:55.418 INFO:teuthology.orchestra.run.smithi002.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-22T00:46:55.419 INFO:teuthology.orchestra.run.smithi002.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-22T00:46:55.419 INFO:teuthology.orchestra.run.smithi002.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-22T00:46:55.420 INFO:teuthology.orchestra.run.smithi002.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-22T00:46:55.420 INFO:teuthology.orchestra.run.smithi002.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-22T00:46:55.434 INFO:teuthology.orchestra.run.smithi119.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-22T00:46:55.439 INFO:teuthology.orchestra.run.smithi119.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-22T00:46:55.445 INFO:teuthology.orchestra.run.smithi119.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-22T00:46:55.445 INFO:teuthology.orchestra.run.smithi119.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-22T00:46:55.446 INFO:teuthology.orchestra.run.smithi119.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-22T00:46:55.488 INFO:teuthology.orchestra.run.smithi002.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-22T00:46:55.489 INFO:teuthology.orchestra.run.smithi002.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-22T00:46:55.491 INFO:teuthology.orchestra.run.smithi002.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-22T00:46:55.491 INFO:teuthology.orchestra.run.smithi002.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-22T00:46:55.492 INFO:teuthology.orchestra.run.smithi002.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-22T00:46:55.493 INFO:teuthology.orchestra.run.smithi002.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-22T00:46:55.495 INFO:teuthology.orchestra.run.smithi119.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-22T00:46:55.495 INFO:teuthology.orchestra.run.smithi119.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-22T00:46:55.522 INFO:teuthology.orchestra.run.smithi119.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-22T00:46:55.522 INFO:teuthology.orchestra.run.smithi119.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-22T00:46:55.524 INFO:teuthology.orchestra.run.smithi119.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-22T00:46:55.525 INFO:teuthology.orchestra.run.smithi119.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-22T00:46:55.525 INFO:teuthology.orchestra.run.smithi119.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-22T00:46:55.526 INFO:teuthology.orchestra.run.smithi119.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-22T00:46:55.527 INFO:teuthology.orchestra.run.smithi119.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-22T00:46:55.572 INFO:teuthology.orchestra.run.smithi002.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-22T00:46:55.573 INFO:teuthology.orchestra.run.smithi002.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-22T00:46:55.573 INFO:teuthology.orchestra.run.smithi002.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-22T00:46:55.574 INFO:teuthology.orchestra.run.smithi002.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-22T00:46:55.575 INFO:teuthology.orchestra.run.smithi002.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-22T00:46:55.607 INFO:teuthology.orchestra.run.smithi119.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-22T00:46:55.608 INFO:teuthology.orchestra.run.smithi119.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-22T00:47:01.474 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 16.3 MB in 3s (5,917 kB/s) 2024-09-22T00:47:01.834 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 16.3 MB in 3s (5,712 kB/s) 2024-09-22T00:47:02.798 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-22T00:47:02.814 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-22T00:47:02.876 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-22T00:47:03.085 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-09-22T00:47:03.086 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-09-22T00:47:03.270 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T00:47:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T00:47:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T00:47:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T00:47:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-22T00:47:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: python2.7-minimal 2024-09-22T00:47:03.271 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T00:47:03.272 INFO:teuthology.orchestra.run.smithi119.stdout:The following additional packages will be installed: 2024-09-22T00:47:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-22T00:47:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-22T00:47:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-22T00:47:03.274 INFO:teuthology.orchestra.run.smithi119.stdout:Suggested packages: 2024-09-22T00:47:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-22T00:47:03.274 INFO:teuthology.orchestra.run.smithi119.stdout:Recommended packages: 2024-09-22T00:47:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: thermald 2024-09-22T00:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2024-09-22T00:47:03.318 INFO:teuthology.orchestra.run.smithi119.stdout:The following NEW packages will be installed: 2024-09-22T00:47:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-22T00:47:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-22T00:47:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-22T00:47:03.319 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages will be upgraded: 2024-09-22T00:47:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-22T00:47:03.323 DEBUG:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-22T00:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2024-09-22T00:47:03.499 INFO:teuthology.orchestra.run.smithi119.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-22T00:47:03.499 INFO:teuthology.orchestra.run.smithi119.stdout:Need to get 113 MB of archives. 2024-09-22T00:47:03.500 INFO:teuthology.orchestra.run.smithi119.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-22T00:47:03.500 INFO:teuthology.orchestra.run.smithi119.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-22T00:47:03.570 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2024-09-22T00:47:03.571 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2024-09-22T00:47:03.743 INFO:teuthology.orchestra.run.smithi002.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T00:47:03.743 INFO:teuthology.orchestra.run.smithi002.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T00:47:03.743 INFO:teuthology.orchestra.run.smithi002.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T00:47:03.743 INFO:teuthology.orchestra.run.smithi002.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T00:47:03.743 INFO:teuthology.orchestra.run.smithi002.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-22T00:47:03.743 INFO:teuthology.orchestra.run.smithi002.stdout: python2.7-minimal 2024-09-22T00:47:03.744 INFO:teuthology.orchestra.run.smithi002.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T00:47:03.745 INFO:teuthology.orchestra.run.smithi002.stdout:The following additional packages will be installed: 2024-09-22T00:47:03.745 INFO:teuthology.orchestra.run.smithi002.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-22T00:47:03.745 INFO:teuthology.orchestra.run.smithi002.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-22T00:47:03.745 INFO:teuthology.orchestra.run.smithi002.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-22T00:47:03.746 INFO:teuthology.orchestra.run.smithi002.stdout:Suggested packages: 2024-09-22T00:47:03.746 INFO:teuthology.orchestra.run.smithi002.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-22T00:47:03.747 INFO:teuthology.orchestra.run.smithi002.stdout:Recommended packages: 2024-09-22T00:47:03.747 INFO:teuthology.orchestra.run.smithi002.stdout: thermald 2024-09-22T00:47:03.786 INFO:teuthology.orchestra.run.smithi002.stdout:The following NEW packages will be installed: 2024-09-22T00:47:03.786 INFO:teuthology.orchestra.run.smithi002.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-22T00:47:03.786 INFO:teuthology.orchestra.run.smithi002.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-22T00:47:03.787 INFO:teuthology.orchestra.run.smithi002.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-22T00:47:03.788 INFO:teuthology.orchestra.run.smithi002.stdout:The following packages will be upgraded: 2024-09-22T00:47:03.789 INFO:teuthology.orchestra.run.smithi002.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-22T00:47:03.841 INFO:teuthology.orchestra.run.smithi002.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-22T00:47:03.841 INFO:teuthology.orchestra.run.smithi002.stdout:Need to get 113 MB of archives. 2024-09-22T00:47:03.841 INFO:teuthology.orchestra.run.smithi002.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-22T00:47:03.841 INFO:teuthology.orchestra.run.smithi002.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-22T00:47:04.323 INFO:teuthology.orchestra.run.smithi002.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-22T00:47:04.420 INFO:teuthology.orchestra.run.smithi002.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-22T00:47:05.000 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-22T00:47:05.257 INFO:teuthology.orchestra.run.smithi002.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-22T00:47:05.257 INFO:teuthology.orchestra.run.smithi002.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-22T00:47:05.257 INFO:teuthology.orchestra.run.smithi002.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-22T00:47:05.354 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-22T00:47:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-22T00:47:05.418 INFO:teuthology.orchestra.run.smithi002.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-22T00:47:05.829 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 113 MB in 2s (69.9 MB/s) 2024-09-22T00:47:05.986 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-22T00:47:07.599 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-22T00:47:07.599 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-22T00:47:07.599 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-22T00:47:08.005 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-22T00:47:08.026 INFO:teuthology.orchestra.run.smithi002.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-22T00:47:08.031 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T00:47:08.089 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-22T00:47:08.267 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:08.479 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 113 MB in 5s (23.8 MB/s) 2024-09-22T00:47:08.647 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-22T00:47:10.284 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-22T00:47:10.305 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T00:47:10.391 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:10.770 INFO:teuthology.orchestra.run.smithi119.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-22T00:47:10.774 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T00:47:10.846 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-22T00:47:10.852 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:10.867 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T00:47:10.904 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:13.272 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-22T00:47:13.293 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T00:47:13.373 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:13.892 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-22T00:47:13.912 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T00:47:14.002 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:16.854 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T00:47:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T00:47:17.357 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T00:47:17.460 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T00:47:17.755 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-22T00:47:17.777 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-22T00:47:17.821 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-22T00:47:20.432 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T00:47:20.551 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T00:47:20.824 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T00:47:20.929 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T00:47:21.248 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-22T00:47:21.270 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-22T00:47:21.315 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-22T00:47:23.069 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-22T00:47:23.106 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T00:47:23.168 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:26.413 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T00:47:26.539 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T00:47:26.873 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-22T00:47:26.994 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:27.038 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-22T00:47:27.075 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T00:47:27.112 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-22T00:47:27.112 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:27.232 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:28.551 INFO:teuthology.orchestra.run.smithi002.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-22T00:47:28.551 INFO:teuthology.orchestra.run.smithi002.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-22T00:47:28.551 INFO:teuthology.orchestra.run.smithi002.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-22T00:47:28.551 INFO:teuthology.orchestra.run.smithi002.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-22T00:47:28.709 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:30.599 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T00:47:30.724 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T00:47:30.727 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-22T00:47:30.844 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-22T00:47:30.945 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:31.066 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-22T00:47:31.187 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:31.313 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-22T00:47:31.459 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:32.372 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:32.430 INFO:teuthology.orchestra.run.smithi002.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-22T00:47:32.431 INFO:teuthology.orchestra.run.smithi002.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-22T00:47:32.791 INFO:teuthology.orchestra.run.smithi119.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-22T00:47:32.791 INFO:teuthology.orchestra.run.smithi119.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-22T00:47:32.792 INFO:teuthology.orchestra.run.smithi119.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-22T00:47:32.792 INFO:teuthology.orchestra.run.smithi119.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-22T00:47:32.952 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:34.361 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-22T00:47:34.487 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-22T00:47:34.613 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:36.624 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T00:47:36.692 INFO:teuthology.orchestra.run.smithi119.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-22T00:47:36.692 INFO:teuthology.orchestra.run.smithi119.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-22T00:47:47.045 INFO:teuthology.orchestra.run.smithi002.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-22T00:47:47.045 INFO:teuthology.orchestra.run.smithi002.stdout:Sourcing file `/etc/default/grub' 2024-09-22T00:47:47.065 INFO:teuthology.orchestra.run.smithi002.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-22T00:47:47.080 INFO:teuthology.orchestra.run.smithi002.stdout:Generating grub configuration file ... 2024-09-22T00:47:47.356 INFO:teuthology.orchestra.run.smithi002.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-22T00:47:47.375 INFO:teuthology.orchestra.run.smithi002.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-22T00:47:47.750 INFO:teuthology.orchestra.run.smithi002.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-22T00:47:47.755 INFO:teuthology.orchestra.run.smithi002.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-22T00:47:47.828 INFO:teuthology.orchestra.run.smithi002.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-22T00:47:47.834 INFO:teuthology.orchestra.run.smithi002.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-22T00:47:47.987 INFO:teuthology.orchestra.run.smithi002.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-22T00:47:47.987 INFO:teuthology.orchestra.run.smithi002.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-22T00:47:47.987 INFO:teuthology.orchestra.run.smithi002.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-22T00:47:48.027 INFO:teuthology.orchestra.run.smithi002.stdout:done 2024-09-22T00:47:48.513 DEBUG:teuthology.orchestra.run.smithi002:> dpkg -s linux-image-generic 2024-09-22T00:47:48.531 INFO:teuthology.orchestra.run.smithi002.stdout:Package: linux-image-generic 2024-09-22T00:47:48.531 INFO:teuthology.orchestra.run.smithi002.stdout:Status: install ok installed 2024-09-22T00:47:48.531 INFO:teuthology.orchestra.run.smithi002.stdout:Priority: optional 2024-09-22T00:47:48.531 INFO:teuthology.orchestra.run.smithi002.stdout:Section: kernel 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Installed-Size: 21 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Maintainer: Ubuntu Kernel Team 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: amd64 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Source: linux-meta 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Version: 5.15.0.122.122 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Recommends: thermald 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:Description: Generic Linux kernel image 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: This package will always depend on the latest generic kernel image 2024-09-22T00:47:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: available. 2024-09-22T00:47:48.532 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-22T00:47:48.533 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T00:47:48.533 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T00:47:48.533 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-22T00:47:48.533 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T00:47:48.533 DEBUG:teuthology.orchestra.run.smithi002:> sudo apt-get clean 2024-09-22T00:47:48.686 DEBUG:teuthology.orchestra.run.smithi002:> sudo apt-get update 2024-09-22T00:47:48.951 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T00:47:48.951 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-22T00:47:49.020 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-22T00:47:49.102 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-22T00:47:50.430 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2024-09-22T00:47:50.449 DEBUG:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-22T00:47:50.491 INFO:teuthology.orchestra.run.smithi119.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-22T00:47:50.491 INFO:teuthology.orchestra.run.smithi119.stdout:Sourcing file `/etc/default/grub' 2024-09-22T00:47:50.502 INFO:teuthology.orchestra.run.smithi119.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-22T00:47:50.514 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2024-09-22T00:47:50.534 INFO:teuthology.orchestra.run.smithi119.stdout:Generating grub configuration file ... 2024-09-22T00:47:50.730 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2024-09-22T00:47:50.730 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2024-09-22T00:47:50.777 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-22T00:47:50.800 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-22T00:47:50.932 INFO:teuthology.orchestra.run.smithi002.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-22T00:47:50.932 INFO:teuthology.orchestra.run.smithi002.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T00:47:50.932 INFO:teuthology.orchestra.run.smithi002.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T00:47:50.932 INFO:teuthology.orchestra.run.smithi002.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T00:47:50.932 INFO:teuthology.orchestra.run.smithi002.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T00:47:50.933 INFO:teuthology.orchestra.run.smithi002.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-22T00:47:50.933 INFO:teuthology.orchestra.run.smithi002.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-22T00:47:50.933 INFO:teuthology.orchestra.run.smithi002.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T00:47:50.976 INFO:teuthology.orchestra.run.smithi002.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-22T00:47:50.978 DEBUG:teuthology.orchestra.run.smithi002:> dpkg -s linux-image-generic 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Package: linux-image-generic 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Status: install ok installed 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Priority: optional 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Section: kernel 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Installed-Size: 21 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Maintainer: Ubuntu Kernel Team 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: amd64 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Source: linux-meta 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Version: 5.15.0.122.122 2024-09-22T00:47:50.994 INFO:teuthology.orchestra.run.smithi002.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-22T00:47:50.995 INFO:teuthology.orchestra.run.smithi002.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-22T00:47:50.995 INFO:teuthology.orchestra.run.smithi002.stdout:Recommends: thermald 2024-09-22T00:47:50.995 INFO:teuthology.orchestra.run.smithi002.stdout:Description: Generic Linux kernel image 2024-09-22T00:47:50.995 INFO:teuthology.orchestra.run.smithi002.stdout: This package will always depend on the latest generic kernel image 2024-09-22T00:47:50.995 INFO:teuthology.orchestra.run.smithi002.stdout: available. 2024-09-22T00:47:50.995 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-22T00:47:51.039 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.4LimJV1iq4 2024-09-22T00:47:51.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub/grub.cfg /tmp/tmp.4LimJV1iq4 2024-09-22T00:47:51.083 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-22T00:47:51.089 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-22T00:47:51.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.4LimJV1iq4 2024-09-22T00:47:51.162 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-22T00:47:51.168 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-22T00:47:51.211 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.4LimJV1iq4 is 10KB 2024-09-22T00:47:51.224 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.4LimJV1iq4 2024-09-22T00:47:51.229 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-22T00:47:51.283 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:47:51.283 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-22T00:47:51.283 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-22T00:47:51.332 INFO:teuthology.orchestra.run.smithi119.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-22T00:47:51.332 INFO:teuthology.orchestra.run.smithi119.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-22T00:47:51.332 INFO:teuthology.orchestra.run.smithi119.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-22T00:47:51.350 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-22T00:47:51.351 DEBUG:teuthology.orchestra.run.smithi002:> sudo update-grub 2024-09-22T00:47:51.375 INFO:teuthology.orchestra.run.smithi119.stdout:done 2024-09-22T00:47:51.851 DEBUG:teuthology.orchestra.run.smithi119:> dpkg -s linux-image-generic 2024-09-22T00:47:51.872 INFO:teuthology.orchestra.run.smithi119.stdout:Package: linux-image-generic 2024-09-22T00:47:51.872 INFO:teuthology.orchestra.run.smithi119.stdout:Status: install ok installed 2024-09-22T00:47:51.872 INFO:teuthology.orchestra.run.smithi119.stdout:Priority: optional 2024-09-22T00:47:51.872 INFO:teuthology.orchestra.run.smithi119.stdout:Section: kernel 2024-09-22T00:47:51.872 INFO:teuthology.orchestra.run.smithi119.stdout:Installed-Size: 21 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Maintainer: Ubuntu Kernel Team 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: amd64 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Source: linux-meta 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Version: 5.15.0.122.122 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Recommends: thermald 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Description: Generic Linux kernel image 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout: This package will always depend on the latest generic kernel image 2024-09-22T00:47:51.873 INFO:teuthology.orchestra.run.smithi119.stdout: available. 2024-09-22T00:47:51.874 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-22T00:47:51.874 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T00:47:51.874 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T00:47:51.874 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-22T00:47:51.874 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T00:47:51.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get clean 2024-09-22T00:47:52.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-09-22T00:47:52.225 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T00:47:52.232 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-22T00:47:52.250 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-22T00:47:52.344 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-22T00:47:52.463 INFO:teuthology.orchestra.run.smithi002.stderr:Sourcing file `/etc/default/grub' 2024-09-22T00:47:52.464 INFO:teuthology.orchestra.run.smithi002.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-22T00:47:52.466 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2024-09-22T00:47:52.635 INFO:teuthology.orchestra.run.smithi002.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-22T00:47:52.642 INFO:teuthology.orchestra.run.smithi002.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-22T00:47:52.869 INFO:teuthology.orchestra.run.smithi002.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-22T00:47:52.874 INFO:teuthology.orchestra.run.smithi002.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-22T00:47:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-22T00:47:52.922 INFO:teuthology.orchestra.run.smithi002.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-22T00:47:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-22T00:47:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-22T00:47:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-22T00:47:53.052 INFO:teuthology.orchestra.run.smithi002.stderr:done 2024-09-22T00:47:53.068 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2024-09-22T00:47:53.673 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-22T00:47:53.691 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-22T00:47:53.752 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-22T00:47:53.963 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-09-22T00:47:53.963 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-09-22T00:47:54.134 INFO:teuthology.orchestra.run.smithi119.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-22T00:47:54.134 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T00:47:54.134 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T00:47:54.134 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T00:47:54.134 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T00:47:54.134 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-22T00:47:54.134 INFO:teuthology.orchestra.run.smithi119.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-22T00:47:54.135 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T00:47:54.173 INFO:teuthology.orchestra.run.smithi119.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-22T00:47:54.175 DEBUG:teuthology.orchestra.run.smithi119:> dpkg -s linux-image-generic 2024-09-22T00:47:54.190 INFO:teuthology.orchestra.run.smithi119.stdout:Package: linux-image-generic 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Status: install ok installed 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Priority: optional 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Section: kernel 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Installed-Size: 21 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Maintainer: Ubuntu Kernel Team 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: amd64 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Source: linux-meta 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Version: 5.15.0.122.122 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Recommends: thermald 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout:Description: Generic Linux kernel image 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout: This package will always depend on the latest generic kernel image 2024-09-22T00:47:54.191 INFO:teuthology.orchestra.run.smithi119.stdout: available. 2024-09-22T00:47:54.192 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-09-22T00:47:54.238 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.cWLyWJge3C 2024-09-22T00:47:54.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub/grub.cfg /tmp/tmp.cWLyWJge3C 2024-09-22T00:47:54.301 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.cWLyWJge3C 2024-09-22T00:47:54.426 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.cWLyWJge3C is 10KB 2024-09-22T00:47:54.439 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.cWLyWJge3C 2024-09-22T00:47:54.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-22T00:47:54.497 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:47:54.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-22T00:47:54.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-22T00:47:54.561 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-22T00:47:54.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo update-grub 2024-09-22T00:47:55.836 INFO:teuthology.orchestra.run.smithi119.stderr:Sourcing file `/etc/default/grub' 2024-09-22T00:47:55.837 INFO:teuthology.orchestra.run.smithi119.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-22T00:47:55.839 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-09-22T00:47:56.027 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-22T00:47:56.033 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-22T00:47:56.268 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-22T00:47:56.273 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-22T00:47:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-22T00:47:56.319 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-22T00:47:56.451 INFO:teuthology.orchestra.run.smithi119.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-22T00:47:56.451 INFO:teuthology.orchestra.run.smithi119.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-22T00:47:56.451 INFO:teuthology.orchestra.run.smithi119.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-22T00:47:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-09-22T00:47:56.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-09-22T00:48:23.097 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-22T00:48:23.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-22T00:48:23.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:48:26.486 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-22T00:48:26.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-22T00:48:26.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:48:41.455 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-09-22T00:48:45.039 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-09-22T00:48:50.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-22T00:48:50.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:48:54.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-22T00:48:54.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:48:57.103 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-09-22T00:49:09.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-22T00:49:09.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:49:27.535 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-09-22T00:49:42.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-22T00:49:42.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:49:42.773 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-09-22T00:49:43.328 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-22T00:49:43.328 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-22T00:49:43.328 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-09-22T00:49:43.374 INFO:teuthology.orchestra.run.smithi119.stdout:5.15.0-122-generic 2024-09-22T00:49:43.374 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-22T00:49:43.374 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T00:49:43.374 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-22T00:49:44.376 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-22T00:49:44.376 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T00:49:44.453 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-09-22T00:49:44.495 DEBUG:teuthology.parallel:result is None 2024-09-22T00:49:50.463 DEBUG:teuthology.orchestra.remote:timed out 2024-09-22T00:50:02.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-22T00:50:02.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:50:02.696 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-09-22T00:50:03.217 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-22T00:50:03.217 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-22T00:50:03.217 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-09-22T00:50:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:5.15.0-122-generic 2024-09-22T00:50:03.264 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-22T00:50:03.264 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T00:50:03.265 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-22T00:50:04.265 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-22T00:50:04.265 DEBUG:teuthology.orchestra.run.smithi002:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T00:50:04.336 INFO:teuthology.orchestra.run.smithi002.stdout:ttyS1 2024-09-22T00:50:04.358 DEBUG:teuthology.parallel:result is None 2024-09-22T00:50:04.358 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-22T00:50:04.372 INFO:teuthology.task.internal:Creating test directory... 2024-09-22T00:50:04.372 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T00:50:04.374 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T00:50:04.380 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-22T00:50:04.393 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-22T00:50:04.406 INFO:teuthology.task.internal:Creating archive directory... 2024-09-22T00:50:04.406 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T00:50:04.421 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T00:50:04.514 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-22T00:50:04.521 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-22T00:50:04.521 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T00:50:04.523 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T00:50:04.543 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T00:50:04.545 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T00:50:04.551 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T00:50:04.555 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T00:50:04.556 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-22T00:50:04.564 INFO:teuthology.task.internal:Configuring sudo... 2024-09-22T00:50:04.564 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T00:50:04.601 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T00:50:04.614 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-22T00:50:04.622 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-22T00:50:04.622 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T00:50:04.657 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T00:50:04.663 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T00:50:04.708 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T00:50:04.756 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:50:04.756 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T00:50:04.824 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T00:50:04.830 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T00:50:04.878 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:50:04.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T00:50:04.934 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2024-09-22T00:50:04.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-09-22T00:50:05.022 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-22T00:50:05.030 INFO:teuthology.task.internal:Starting timer... 2024-09-22T00:50:05.030 INFO:teuthology.run_tasks:Running task pcp... 2024-09-22T00:50:05.040 INFO:teuthology.run_tasks:Running task selinux... 2024-09-22T00:50:05.048 DEBUG:teuthology.task.selinux:Excluding smithi002: OS 'ubuntu' does not support SELinux 2024-09-22T00:50:05.049 DEBUG:teuthology.task.selinux:Excluding smithi119: OS 'ubuntu' does not support SELinux 2024-09-22T00:50:05.049 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-22T00:50:05.049 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-22T00:50:05.049 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-22T00:50:05.049 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-22T00:50:05.058 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-22T00:50:05.068 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-22T00:50:05.069 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi119.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-22T00:57:40.558 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com')] 2024-09-22T00:57:40.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-22T00:57:40.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:57:40.642 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-09-22T00:57:40.720 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-22T00:57:40.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-22T00:57:40.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T00:57:40.800 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-09-22T00:57:40.882 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-22T00:57:40.882 INFO:teuthology.run_tasks:Running task clock... 2024-09-22T00:57:40.892 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-22T00:57:40.892 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T00:57:40.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T00:57:40.895 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T00:57:40.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T00:57:40.923 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-22T00:57:40.923 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Command line: ntpd -gq 2024-09-22T00:57:40.923 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: ---------------------------------------------------- 2024-09-22T00:57:40.923 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: ntp-4 is maintained by Network Time Foundation, 2024-09-22T00:57:40.923 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-22T00:57:40.923 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: corporation. Support and training for ntp-4 are 2024-09-22T00:57:40.923 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: available at https://www.nwtime.org/support 2024-09-22T00:57:40.923 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: ---------------------------------------------------- 2024-09-22T00:57:40.924 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: proto: precision = 0.069 usec (-24) 2024-09-22T00:57:40.924 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: basedate set to 2022-02-04 2024-09-22T00:57:40.924 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: gps base set to 2022-02-06 (week 2196) 2024-09-22T00:57:40.924 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-22T00:57:40.925 INFO:teuthology.orchestra.run.smithi002.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-22T00:57:40.925 INFO:teuthology.orchestra.run.smithi002.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-22T00:57:40.925 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: restrict ::: KOD does nothing without LIMITED. 2024-09-22T00:57:40.926 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Listen and drop on 0 v6wildcard [::]:123 2024-09-22T00:57:40.926 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-22T00:57:40.926 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Listen normally on 2 lo 127.0.0.1:123 2024-09-22T00:57:40.926 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Listen normally on 3 ens1f0 172.21.15.2:123 2024-09-22T00:57:40.926 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Listen normally on 4 lo [::1]:123 2024-09-22T00:57:40.927 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1920%4]:123 2024-09-22T00:57:40.927 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:40 ntpd[18361]: Listening on routing socket on fd #22 for interface updates 2024-09-22T00:57:40.953 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-22T00:57:40.953 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Command line: ntpd -gq 2024-09-22T00:57:40.953 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: ---------------------------------------------------- 2024-09-22T00:57:40.953 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: ntp-4 is maintained by Network Time Foundation, 2024-09-22T00:57:40.953 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-22T00:57:40.953 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: corporation. Support and training for ntp-4 are 2024-09-22T00:57:40.953 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: available at https://www.nwtime.org/support 2024-09-22T00:57:40.953 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: ---------------------------------------------------- 2024-09-22T00:57:40.954 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: proto: precision = 0.046 usec (-24) 2024-09-22T00:57:40.954 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: basedate set to 2022-02-04 2024-09-22T00:57:40.954 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: gps base set to 2022-02-06 (week 2196) 2024-09-22T00:57:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-22T00:57:40.955 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-22T00:57:40.955 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: restrict ::: KOD does nothing without LIMITED. 2024-09-22T00:57:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-22T00:57:40.956 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Listen and drop on 0 v6wildcard [::]:123 2024-09-22T00:57:40.956 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-22T00:57:40.956 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Listen normally on 2 lo 127.0.0.1:123 2024-09-22T00:57:40.956 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Listen normally on 3 enp3s0f1 172.21.15.119:123 2024-09-22T00:57:40.956 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Listen normally on 4 lo [::1]:123 2024-09-22T00:57:40.956 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d5ef%5]:123 2024-09-22T00:57:40.956 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:40 ntpd[18524]: Listening on routing socket on fd #22 for interface updates 2024-09-22T00:57:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:41 ntpd[18524]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:42.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:42.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:42.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:42 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:42.953 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:42 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:42.953 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:42 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:43 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:43 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:43 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:43 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:43 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:43 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:43 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:43 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:43.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:43.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:43.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:43.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:43.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:43.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:43 ntpd[18524]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:44 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:45.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:45 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:45.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:45 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:45.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:45 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:45.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:45 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:45.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:45 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:45.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:45 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:45.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:45.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:45.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:45.956 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:45.956 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:45 ntpd[18524]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:46.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:46.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:46.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:46.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:46 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:47 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:47 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:47 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:47 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:47 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:47 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:47.954 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:47.955 INFO:teuthology.orchestra.run.smithi119.stdout:22 Sep 00:57:47 ntpd[18524]: ntpd: time slew -0.001996 s 2024-09-22T00:57:47.955 INFO:teuthology.orchestra.run.smithi119.stdout:ntpd: time slew -0.001996s 2024-09-22T00:57:47.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-22T00:57:47.955 INFO:teuthology.orchestra.run.smithi119.stderr:22 Sep 00:57:47 ntpd[18524]: can't open /var/log/ntpstats/loopstats.20240922: Permission denied 2024-09-22T00:57:48.004 INFO:teuthology.orchestra.run.smithi119.stdout: remote refid st t when poll reach delay offset jitter 2024-09-22T00:57:48.004 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================== 2024-09-22T00:57:48.004 INFO:teuthology.orchestra.run.smithi119.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T00:57:48.004 INFO:teuthology.orchestra.run.smithi119.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T00:57:48.004 INFO:teuthology.orchestra.run.smithi119.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T00:57:48.004 INFO:teuthology.orchestra.run.smithi119.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T00:57:48.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:48.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:48.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:48.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:48.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T00:57:48.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T00:57:48.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T00:57:48.924 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T00:57:48.925 INFO:teuthology.orchestra.run.smithi002.stdout:22 Sep 00:57:48 ntpd[18361]: ntpd: time slew -0.000877 s 2024-09-22T00:57:48.925 INFO:teuthology.orchestra.run.smithi002.stdout:ntpd: time slew -0.000877s 2024-09-22T00:57:48.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-22T00:57:48.925 INFO:teuthology.orchestra.run.smithi002.stderr:22 Sep 00:57:48 ntpd[18361]: can't open /var/log/ntpstats/loopstats.20240922: Permission denied 2024-09-22T00:57:48.974 INFO:teuthology.orchestra.run.smithi002.stdout: remote refid st t when poll reach delay offset jitter 2024-09-22T00:57:48.974 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================== 2024-09-22T00:57:48.974 INFO:teuthology.orchestra.run.smithi002.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T00:57:48.974 INFO:teuthology.orchestra.run.smithi002.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T00:57:48.974 INFO:teuthology.orchestra.run.smithi002.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T00:57:48.974 INFO:teuthology.orchestra.run.smithi002.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T00:57:48.974 INFO:teuthology.run_tasks:Running task install... 2024-09-22T00:57:48.982 DEBUG:teuthology.task.install:project ceph 2024-09-22T00:57:48.982 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-22T00:57:48.982 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-22T00:57:48.982 INFO:teuthology.task.install:Using flavor: default 2024-09-22T00:57:48.988 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-22T00:57:48.988 INFO:teuthology.task.install:extra packages: [] 2024-09-22T00:57:48.988 DEBUG:teuthology.orchestra.run.smithi002:> sudo apt-key list | grep Ceph 2024-09-22T00:57:48.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-key list | grep Ceph 2024-09-22T00:57:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-22T00:57:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-22T00:57:49.078 INFO:teuthology.orchestra.run.smithi119.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-22T00:57:49.079 INFO:teuthology.orchestra.run.smithi119.stdout:uid [ unknown] Ceph.com (release key) 2024-09-22T00:57:49.079 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-22T00:57:49.079 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T00:57:49.111 INFO:teuthology.orchestra.run.smithi002.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-22T00:57:49.112 INFO:teuthology.orchestra.run.smithi002.stdout:uid [ unknown] Ceph.com (release key) 2024-09-22T00:57:49.112 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-22T00:57:49.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T00:57:49.215 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-22T00:57:49.215 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:57:49.249 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-22T00:57:49.250 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:57:49.325 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:57:49.325 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-22T00:57:49.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-09-22T00:57:49.355 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:57:49.355 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-22T00:57:49.368 DEBUG:teuthology.orchestra.run.smithi002:> sudo apt-get update 2024-09-22T00:57:49.540 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T00:57:49.549 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-22T00:57:49.561 INFO:teuthology.orchestra.run.smithi119.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-22T00:57:49.566 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-22T00:57:49.582 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-22T00:57:49.595 INFO:teuthology.orchestra.run.smithi002.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-22T00:57:49.605 INFO:teuthology.orchestra.run.smithi119.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-22T00:57:49.616 INFO:teuthology.orchestra.run.smithi002.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-22T00:57:49.626 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-22T00:57:49.637 INFO:teuthology.orchestra.run.smithi002.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-22T00:57:49.644 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T00:57:49.644 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-22T00:57:49.659 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-22T00:57:49.664 INFO:teuthology.orchestra.run.smithi002.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-22T00:57:49.665 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-22T00:57:49.686 INFO:teuthology.orchestra.run.smithi002.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-22T00:57:49.708 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-22T00:57:49.789 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-22T00:57:49.896 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 28.9 kB in 0s (83.9 kB/s) 2024-09-22T00:57:50.007 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 28.9 kB in 0s (59.1 kB/s) 2024-09-22T00:57:50.951 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2024-09-22T00:57:50.969 DEBUG:teuthology.orchestra.run.smithi002:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:57:51.023 INFO:teuthology.orchestra.run.smithi002.stdout:Reading package lists... 2024-09-22T00:57:51.030 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-22T00:57:51.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:57:51.110 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-22T00:57:51.214 INFO:teuthology.orchestra.run.smithi002.stdout:Building dependency tree... 2024-09-22T00:57:51.214 INFO:teuthology.orchestra.run.smithi002.stdout:Reading state information... 2024-09-22T00:57:51.320 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-09-22T00:57:51.321 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-09-22T00:57:51.408 INFO:teuthology.orchestra.run.smithi002.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T00:57:51.408 INFO:teuthology.orchestra.run.smithi002.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T00:57:51.409 INFO:teuthology.orchestra.run.smithi002.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T00:57:51.409 INFO:teuthology.orchestra.run.smithi002.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T00:57:51.409 INFO:teuthology.orchestra.run.smithi002.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-22T00:57:51.409 INFO:teuthology.orchestra.run.smithi002.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-22T00:57:51.409 INFO:teuthology.orchestra.run.smithi002.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T00:57:51.410 INFO:teuthology.orchestra.run.smithi002.stdout:The following additional packages will be installed: 2024-09-22T00:57:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-22T00:57:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-22T00:57:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-22T00:57:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-22T00:57:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet zip 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout:Suggested packages: 2024-09-22T00:57:51.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-22T00:57:51.414 INFO:teuthology.orchestra.run.smithi002.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-22T00:57:51.414 INFO:teuthology.orchestra.run.smithi002.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-22T00:57:51.414 INFO:teuthology.orchestra.run.smithi002.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-22T00:57:51.414 INFO:teuthology.orchestra.run.smithi002.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-22T00:57:51.414 INFO:teuthology.orchestra.run.smithi002.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-22T00:57:51.414 INFO:teuthology.orchestra.run.smithi002.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-22T00:57:51.414 INFO:teuthology.orchestra.run.smithi002.stdout:Recommended packages: 2024-09-22T00:57:51.414 INFO:teuthology.orchestra.run.smithi002.stdout: btrfs-tools 2024-09-22T00:57:51.517 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T00:57:51.517 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T00:57:51.517 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T00:57:51.518 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T00:57:51.519 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-22T00:57:51.519 INFO:teuthology.orchestra.run.smithi119.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-22T00:57:51.519 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout:The following NEW packages will be installed: 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-22T00:57:51.520 INFO:teuthology.orchestra.run.smithi002.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-22T00:57:51.521 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-22T00:57:51.522 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-22T00:57:51.522 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-22T00:57:51.522 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-22T00:57:51.522 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-22T00:57:51.522 INFO:teuthology.orchestra.run.smithi002.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-22T00:57:51.523 INFO:teuthology.orchestra.run.smithi119.stdout:The following additional packages will be installed: 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-22T00:57:51.524 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet zip 2024-09-22T00:57:51.525 INFO:teuthology.orchestra.run.smithi119.stdout:Suggested packages: 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout:Recommended packages: 2024-09-22T00:57:51.526 INFO:teuthology.orchestra.run.smithi119.stdout: btrfs-tools 2024-09-22T00:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout:0 upgraded, 118 newly installed, 0 to remove and 332 not upgraded. 2024-09-22T00:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout:Need to get 181 MB of archives. 2024-09-22T00:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-22T00:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-22T00:57:51.623 INFO:teuthology.orchestra.run.smithi119.stdout:The following NEW packages will be installed: 2024-09-22T00:57:51.623 INFO:teuthology.orchestra.run.smithi119.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-22T00:57:51.623 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-22T00:57:51.623 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-22T00:57:51.625 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-22T00:57:51.625 INFO:teuthology.orchestra.run.smithi119.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-22T00:57:51.625 INFO:teuthology.orchestra.run.smithi119.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-22T00:57:51.625 INFO:teuthology.orchestra.run.smithi119.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-22T00:57:51.625 INFO:teuthology.orchestra.run.smithi119.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-22T00:57:51.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-22T00:57:51.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-22T00:57:51.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-22T00:57:51.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-22T00:57:51.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-22T00:57:51.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-22T00:57:51.627 INFO:teuthology.orchestra.run.smithi119.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-22T00:57:51.665 INFO:teuthology.orchestra.run.smithi002.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-22T00:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-22T00:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-22T00:57:51.684 INFO:teuthology.orchestra.run.smithi119.stdout:0 upgraded, 118 newly installed, 0 to remove and 332 not upgraded. 2024-09-22T00:57:51.684 INFO:teuthology.orchestra.run.smithi119.stdout:Need to get 181 MB of archives. 2024-09-22T00:57:51.685 INFO:teuthology.orchestra.run.smithi119.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-22T00:57:51.685 INFO:teuthology.orchestra.run.smithi119.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-22T00:57:51.688 INFO:teuthology.orchestra.run.smithi002.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-22T00:57:51.753 INFO:teuthology.orchestra.run.smithi002.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-22T00:57:51.758 INFO:teuthology.orchestra.run.smithi002.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-22T00:57:51.767 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-22T00:57:51.774 INFO:teuthology.orchestra.run.smithi002.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-22T00:57:51.777 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-22T00:57:51.779 INFO:teuthology.orchestra.run.smithi002.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-22T00:57:51.779 INFO:teuthology.orchestra.run.smithi002.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-22T00:57:51.780 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-22T00:57:51.780 INFO:teuthology.orchestra.run.smithi002.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-22T00:57:51.790 INFO:teuthology.orchestra.run.smithi002.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-22T00:57:51.793 INFO:teuthology.orchestra.run.smithi002.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-22T00:57:51.796 INFO:teuthology.orchestra.run.smithi002.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-22T00:57:51.796 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-22T00:57:51.797 INFO:teuthology.orchestra.run.smithi002.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-22T00:57:51.798 INFO:teuthology.orchestra.run.smithi002.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-22T00:57:51.809 INFO:teuthology.orchestra.run.smithi002.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-22T00:57:51.812 INFO:teuthology.orchestra.run.smithi002.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-22T00:57:51.814 INFO:teuthology.orchestra.run.smithi002.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-22T00:57:51.815 INFO:teuthology.orchestra.run.smithi002.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-22T00:57:51.827 INFO:teuthology.orchestra.run.smithi002.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-22T00:57:51.828 INFO:teuthology.orchestra.run.smithi002.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-22T00:57:51.828 INFO:teuthology.orchestra.run.smithi002.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-22T00:57:51.829 INFO:teuthology.orchestra.run.smithi002.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-22T00:57:51.829 INFO:teuthology.orchestra.run.smithi002.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-22T00:57:51.829 INFO:teuthology.orchestra.run.smithi002.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-22T00:57:51.844 INFO:teuthology.orchestra.run.smithi002.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-22T00:57:51.844 INFO:teuthology.orchestra.run.smithi002.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-22T00:57:51.847 INFO:teuthology.orchestra.run.smithi002.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-22T00:57:51.848 INFO:teuthology.orchestra.run.smithi002.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-22T00:57:51.849 INFO:teuthology.orchestra.run.smithi002.stdout:Get:31 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-22T00:57:51.856 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-22T00:57:51.861 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-22T00:57:51.861 INFO:teuthology.orchestra.run.smithi002.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-22T00:57:51.862 INFO:teuthology.orchestra.run.smithi002.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-22T00:57:51.862 INFO:teuthology.orchestra.run.smithi002.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-22T00:57:51.863 INFO:teuthology.orchestra.run.smithi002.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-22T00:57:51.863 INFO:teuthology.orchestra.run.smithi002.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-22T00:57:51.865 INFO:teuthology.orchestra.run.smithi002.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-22T00:57:51.873 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-22T00:57:51.877 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-22T00:57:51.878 INFO:teuthology.orchestra.run.smithi119.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-22T00:57:51.878 INFO:teuthology.orchestra.run.smithi002.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-22T00:57:51.878 INFO:teuthology.orchestra.run.smithi119.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-22T00:57:51.879 INFO:teuthology.orchestra.run.smithi002.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-22T00:57:51.884 INFO:teuthology.orchestra.run.smithi002.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-22T00:57:51.885 INFO:teuthology.orchestra.run.smithi002.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-22T00:57:51.887 INFO:teuthology.orchestra.run.smithi119.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-22T00:57:51.889 INFO:teuthology.orchestra.run.smithi119.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-22T00:57:51.891 INFO:teuthology.orchestra.run.smithi119.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-22T00:57:51.893 INFO:teuthology.orchestra.run.smithi119.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-22T00:57:51.895 INFO:teuthology.orchestra.run.smithi119.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-22T00:57:51.896 INFO:teuthology.orchestra.run.smithi002.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-22T00:57:51.897 INFO:teuthology.orchestra.run.smithi002.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-22T00:57:51.897 INFO:teuthology.orchestra.run.smithi119.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-22T00:57:51.899 INFO:teuthology.orchestra.run.smithi002.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-22T00:57:51.901 INFO:teuthology.orchestra.run.smithi002.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-22T00:57:51.902 INFO:teuthology.orchestra.run.smithi002.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-22T00:57:51.903 INFO:teuthology.orchestra.run.smithi002.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-22T00:57:51.910 INFO:teuthology.orchestra.run.smithi119.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-22T00:57:51.912 INFO:teuthology.orchestra.run.smithi119.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-22T00:57:51.913 INFO:teuthology.orchestra.run.smithi002.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-22T00:57:51.913 INFO:teuthology.orchestra.run.smithi119.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-22T00:57:51.913 INFO:teuthology.orchestra.run.smithi119.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-22T00:57:51.914 INFO:teuthology.orchestra.run.smithi119.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-22T00:57:51.915 INFO:teuthology.orchestra.run.smithi119.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-22T00:57:51.915 INFO:teuthology.orchestra.run.smithi002.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-22T00:57:51.915 INFO:teuthology.orchestra.run.smithi119.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-22T00:57:51.916 INFO:teuthology.orchestra.run.smithi119.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-22T00:57:51.917 INFO:teuthology.orchestra.run.smithi002.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-22T00:57:51.918 INFO:teuthology.orchestra.run.smithi002.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-22T00:57:51.919 INFO:teuthology.orchestra.run.smithi002.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-22T00:57:51.925 INFO:teuthology.orchestra.run.smithi002.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-22T00:57:51.930 INFO:teuthology.orchestra.run.smithi119.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-22T00:57:51.930 INFO:teuthology.orchestra.run.smithi119.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-22T00:57:51.932 INFO:teuthology.orchestra.run.smithi002.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-22T00:57:51.932 INFO:teuthology.orchestra.run.smithi002.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-22T00:57:51.933 INFO:teuthology.orchestra.run.smithi002.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-22T00:57:51.937 INFO:teuthology.orchestra.run.smithi002.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-22T00:57:51.939 INFO:teuthology.orchestra.run.smithi002.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-22T00:57:51.944 INFO:teuthology.orchestra.run.smithi002.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-22T00:57:51.947 INFO:teuthology.orchestra.run.smithi119.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-22T00:57:51.950 INFO:teuthology.orchestra.run.smithi119.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-22T00:57:51.950 INFO:teuthology.orchestra.run.smithi119.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-22T00:57:51.951 INFO:teuthology.orchestra.run.smithi119.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-22T00:57:51.951 INFO:teuthology.orchestra.run.smithi119.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-22T00:57:51.952 INFO:teuthology.orchestra.run.smithi119.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-22T00:57:51.953 INFO:teuthology.orchestra.run.smithi119.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-22T00:57:51.953 INFO:teuthology.orchestra.run.smithi119.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-22T00:57:51.964 INFO:teuthology.orchestra.run.smithi119.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-22T00:57:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-22T00:57:51.982 INFO:teuthology.orchestra.run.smithi119.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-22T00:57:51.987 INFO:teuthology.orchestra.run.smithi119.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-22T00:57:51.987 INFO:teuthology.orchestra.run.smithi119.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-22T00:57:51.988 INFO:teuthology.orchestra.run.smithi119.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-22T00:57:51.988 INFO:teuthology.orchestra.run.smithi119.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-22T00:57:51.989 INFO:teuthology.orchestra.run.smithi119.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-22T00:57:51.990 INFO:teuthology.orchestra.run.smithi002.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-22T00:57:51.991 INFO:teuthology.orchestra.run.smithi002.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-22T00:57:51.991 INFO:teuthology.orchestra.run.smithi119.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-22T00:57:51.992 INFO:teuthology.orchestra.run.smithi119.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-22T00:57:51.993 INFO:teuthology.orchestra.run.smithi002.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-22T00:57:51.993 INFO:teuthology.orchestra.run.smithi002.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-22T00:57:51.994 INFO:teuthology.orchestra.run.smithi002.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-22T00:57:51.998 INFO:teuthology.orchestra.run.smithi119.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-22T00:57:51.999 INFO:teuthology.orchestra.run.smithi119.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-22T00:57:52.012 INFO:teuthology.orchestra.run.smithi002.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-22T00:57:52.014 INFO:teuthology.orchestra.run.smithi002.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-22T00:57:52.014 INFO:teuthology.orchestra.run.smithi002.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-22T00:57:52.016 INFO:teuthology.orchestra.run.smithi119.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-22T00:57:52.016 INFO:teuthology.orchestra.run.smithi119.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-22T00:57:52.017 INFO:teuthology.orchestra.run.smithi119.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-22T00:57:52.019 INFO:teuthology.orchestra.run.smithi119.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-22T00:57:52.028 INFO:teuthology.orchestra.run.smithi002.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-22T00:57:52.029 INFO:teuthology.orchestra.run.smithi002.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-22T00:57:52.029 INFO:teuthology.orchestra.run.smithi002.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-22T00:57:52.030 INFO:teuthology.orchestra.run.smithi002.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-22T00:57:52.030 INFO:teuthology.orchestra.run.smithi002.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-22T00:57:52.030 INFO:teuthology.orchestra.run.smithi002.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-22T00:57:52.044 INFO:teuthology.orchestra.run.smithi002.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-22T00:57:52.045 INFO:teuthology.orchestra.run.smithi002.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-22T00:57:52.049 INFO:teuthology.orchestra.run.smithi002.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-22T00:57:52.062 INFO:teuthology.orchestra.run.smithi002.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-22T00:57:52.063 INFO:teuthology.orchestra.run.smithi002.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-22T00:57:52.063 INFO:teuthology.orchestra.run.smithi002.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-22T00:57:52.066 INFO:teuthology.orchestra.run.smithi002.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-22T00:57:52.069 INFO:teuthology.orchestra.run.smithi002.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-22T00:57:52.069 INFO:teuthology.orchestra.run.smithi002.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-22T00:57:52.070 INFO:teuthology.orchestra.run.smithi002.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-22T00:57:52.072 INFO:teuthology.orchestra.run.smithi119.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-22T00:57:52.073 INFO:teuthology.orchestra.run.smithi119.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-22T00:57:52.079 INFO:teuthology.orchestra.run.smithi002.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-22T00:57:52.079 INFO:teuthology.orchestra.run.smithi002.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-22T00:57:52.082 INFO:teuthology.orchestra.run.smithi002.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-22T00:57:52.085 INFO:teuthology.orchestra.run.smithi119.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-22T00:57:52.086 INFO:teuthology.orchestra.run.smithi119.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-22T00:57:52.087 INFO:teuthology.orchestra.run.smithi119.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-22T00:57:52.088 INFO:teuthology.orchestra.run.smithi002.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-22T00:57:52.090 INFO:teuthology.orchestra.run.smithi002.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-22T00:57:52.096 INFO:teuthology.orchestra.run.smithi002.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-22T00:57:52.097 INFO:teuthology.orchestra.run.smithi002.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-22T00:57:52.098 INFO:teuthology.orchestra.run.smithi002.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-22T00:57:52.098 INFO:teuthology.orchestra.run.smithi002.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-22T00:57:52.098 INFO:teuthology.orchestra.run.smithi002.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-22T00:57:52.099 INFO:teuthology.orchestra.run.smithi002.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-22T00:57:52.104 INFO:teuthology.orchestra.run.smithi119.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-22T00:57:52.105 INFO:teuthology.orchestra.run.smithi002.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-22T00:57:52.105 INFO:teuthology.orchestra.run.smithi002.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-22T00:57:52.106 INFO:teuthology.orchestra.run.smithi119.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-22T00:57:52.106 INFO:teuthology.orchestra.run.smithi119.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-22T00:57:52.113 INFO:teuthology.orchestra.run.smithi002.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-22T00:57:52.115 INFO:teuthology.orchestra.run.smithi002.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-22T00:57:52.116 INFO:teuthology.orchestra.run.smithi002.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-22T00:57:52.122 INFO:teuthology.orchestra.run.smithi119.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-22T00:57:52.123 INFO:teuthology.orchestra.run.smithi119.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-22T00:57:52.123 INFO:teuthology.orchestra.run.smithi119.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-22T00:57:52.123 INFO:teuthology.orchestra.run.smithi119.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-22T00:57:52.124 INFO:teuthology.orchestra.run.smithi119.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-22T00:57:52.124 INFO:teuthology.orchestra.run.smithi119.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-22T00:57:52.137 INFO:teuthology.orchestra.run.smithi119.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-22T00:57:52.137 INFO:teuthology.orchestra.run.smithi119.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-22T00:57:52.137 INFO:teuthology.orchestra.run.smithi119.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-22T00:57:52.141 INFO:teuthology.orchestra.run.smithi119.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-22T00:57:52.154 INFO:teuthology.orchestra.run.smithi119.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-22T00:57:52.155 INFO:teuthology.orchestra.run.smithi119.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-22T00:57:52.156 INFO:teuthology.orchestra.run.smithi119.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-22T00:57:52.159 INFO:teuthology.orchestra.run.smithi119.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-22T00:57:52.161 INFO:teuthology.orchestra.run.smithi119.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-22T00:57:52.162 INFO:teuthology.orchestra.run.smithi119.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-22T00:57:52.162 INFO:teuthology.orchestra.run.smithi119.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-22T00:57:52.171 INFO:teuthology.orchestra.run.smithi119.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-22T00:57:52.172 INFO:teuthology.orchestra.run.smithi119.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-22T00:57:52.174 INFO:teuthology.orchestra.run.smithi119.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-22T00:57:52.188 INFO:teuthology.orchestra.run.smithi119.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-22T00:57:52.189 INFO:teuthology.orchestra.run.smithi119.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-22T00:57:52.190 INFO:teuthology.orchestra.run.smithi119.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-22T00:57:52.191 INFO:teuthology.orchestra.run.smithi119.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-22T00:57:52.192 INFO:teuthology.orchestra.run.smithi119.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-22T00:57:52.198 INFO:teuthology.orchestra.run.smithi119.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-22T00:57:52.198 INFO:teuthology.orchestra.run.smithi119.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-22T00:57:52.205 INFO:teuthology.orchestra.run.smithi119.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-22T00:57:52.207 INFO:teuthology.orchestra.run.smithi119.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-22T00:57:52.208 INFO:teuthology.orchestra.run.smithi119.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-22T00:57:52.295 INFO:teuthology.orchestra.run.smithi119.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-22T00:57:52.355 INFO:teuthology.orchestra.run.smithi119.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-22T00:57:52.362 INFO:teuthology.orchestra.run.smithi119.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-22T00:57:52.363 INFO:teuthology.orchestra.run.smithi119.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-22T00:57:52.367 INFO:teuthology.orchestra.run.smithi119.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-22T00:57:52.369 INFO:teuthology.orchestra.run.smithi119.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-22T00:57:52.420 INFO:teuthology.orchestra.run.smithi119.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-22T00:57:52.737 INFO:teuthology.orchestra.run.smithi119.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-22T00:57:52.768 INFO:teuthology.orchestra.run.smithi119.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-22T00:57:52.778 INFO:teuthology.orchestra.run.smithi119.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-22T00:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-22T00:57:52.876 INFO:teuthology.orchestra.run.smithi002.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-22T00:57:52.880 INFO:teuthology.orchestra.run.smithi002.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-22T00:57:52.883 INFO:teuthology.orchestra.run.smithi002.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-22T00:57:52.918 INFO:teuthology.orchestra.run.smithi002.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-22T00:57:53.144 INFO:teuthology.orchestra.run.smithi002.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-22T00:57:53.557 INFO:teuthology.orchestra.run.smithi119.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-22T00:57:53.641 INFO:teuthology.orchestra.run.smithi119.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-22T00:57:53.645 INFO:teuthology.orchestra.run.smithi119.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-22T00:57:53.647 INFO:teuthology.orchestra.run.smithi119.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-22T00:57:53.700 INFO:teuthology.orchestra.run.smithi119.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-22T00:57:53.851 INFO:teuthology.orchestra.run.smithi119.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-22T00:57:54.220 INFO:teuthology.orchestra.run.smithi002.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-22T00:57:54.220 INFO:teuthology.orchestra.run.smithi002.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-22T00:57:54.238 INFO:teuthology.orchestra.run.smithi002.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-22T00:57:54.310 INFO:teuthology.orchestra.run.smithi002.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-22T00:57:54.324 INFO:teuthology.orchestra.run.smithi002.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-22T00:57:54.327 INFO:teuthology.orchestra.run.smithi002.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-22T00:57:54.393 INFO:teuthology.orchestra.run.smithi002.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-22T00:57:54.585 INFO:teuthology.orchestra.run.smithi119.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-22T00:57:54.586 INFO:teuthology.orchestra.run.smithi119.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-22T00:57:54.603 INFO:teuthology.orchestra.run.smithi119.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-22T00:57:54.645 INFO:teuthology.orchestra.run.smithi119.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-22T00:57:54.655 INFO:teuthology.orchestra.run.smithi119.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-22T00:57:54.656 INFO:teuthology.orchestra.run.smithi119.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-22T00:57:54.663 INFO:teuthology.orchestra.run.smithi002.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-22T00:57:54.663 INFO:teuthology.orchestra.run.smithi002.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-22T00:57:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-22T00:57:54.972 INFO:teuthology.orchestra.run.smithi119.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-22T00:57:54.972 INFO:teuthology.orchestra.run.smithi119.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-22T00:57:56.658 INFO:teuthology.orchestra.run.smithi002.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-22T00:57:56.660 INFO:teuthology.orchestra.run.smithi002.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-22T00:57:56.661 INFO:teuthology.orchestra.run.smithi002.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-22T00:57:56.813 INFO:teuthology.orchestra.run.smithi119.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-22T00:57:56.816 INFO:teuthology.orchestra.run.smithi119.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-22T00:57:56.817 INFO:teuthology.orchestra.run.smithi119.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-22T00:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-22T00:57:57.013 INFO:teuthology.orchestra.run.smithi119.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-22T00:57:57.464 INFO:teuthology.orchestra.run.smithi002.stdout:Fetched 181 MB in 5s (34.0 MB/s) 2024-09-22T00:57:57.581 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 181 MB in 5s (33.7 MB/s) 2024-09-22T00:57:57.618 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-22T00:57:57.721 INFO:teuthology.orchestra.run.smithi002.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-22T00:57:57.726 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-22T00:57:57.792 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-22T00:57:57.808 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-22T00:57:57.894 INFO:teuthology.orchestra.run.smithi119.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-22T00:57:57.899 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-22T00:57:57.998 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-22T00:57:58.145 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-22T00:57:58.169 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-22T00:57:58.211 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-22T00:57:58.302 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-22T00:57:58.328 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-22T00:57:58.368 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-22T00:57:58.539 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-22T00:57:58.566 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-22T00:57:58.606 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-22T00:57:58.738 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-22T00:57:58.764 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-22T00:57:58.805 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-22T00:57:59.001 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-22T00:57:59.028 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T00:57:59.068 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:57:59.258 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-22T00:57:59.285 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T00:57:59.333 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:57:59.454 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-22T00:57:59.481 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T00:57:59.520 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:57:59.745 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-22T00:57:59.771 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T00:57:59.811 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:57:59.865 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-22T00:57:59.892 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T00:57:59.932 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-22T00:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-22T00:58:00.293 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-22T00:58:00.398 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-22T00:58:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T00:58:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:58:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package librados2. 2024-09-22T00:58:00.614 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:00.654 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:00.784 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-22T00:58:00.810 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-22T00:58:00.851 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-22T00:58:01.107 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package librbd1. 2024-09-22T00:58:01.134 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librados2. 2024-09-22T00:58:01.173 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:01.237 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:01.601 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libcephfs2. 2024-09-22T00:58:01.628 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:01.667 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:01.740 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librbd1. 2024-09-22T00:58:01.766 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:01.806 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:01.937 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-rados. 2024-09-22T00:58:01.953 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:01.987 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:02.259 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libcephfs2. 2024-09-22T00:58:02.273 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-22T00:58:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:02.300 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:02.326 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:02.494 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:02.612 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rados. 2024-09-22T00:58:02.638 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:02.678 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:02.817 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cephfs. 2024-09-22T00:58:02.844 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:02.884 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:03.015 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-22T00:58:03.041 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:03.081 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:03.186 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-22T00:58:03.213 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:03.317 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cephfs. 2024-09-22T00:58:03.332 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:03.367 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:03.581 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-22T00:58:03.608 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-22T00:58:03.648 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-22T00:58:03.678 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-22T00:58:03.704 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:03.744 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:03.925 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-prettytable. 2024-09-22T00:58:03.941 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-22T00:58:03.984 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-22T00:58:04.073 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-22T00:58:04.099 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-22T00:58:04.139 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-22T00:58:04.245 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-rbd. 2024-09-22T00:58:04.262 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:04.303 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:04.467 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-prettytable. 2024-09-22T00:58:04.493 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-22T00:58:04.534 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-22T00:58:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-22T00:58:04.641 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-22T00:58:04.672 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-22T00:58:04.795 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rbd. 2024-09-22T00:58:04.821 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:04.861 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:04.941 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-22T00:58:04.968 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-22T00:58:05.008 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-22T00:58:05.223 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-22T00:58:05.249 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-22T00:58:05.298 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-22T00:58:05.302 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-22T00:58:05.329 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-22T00:58:05.369 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-22T00:58:05.672 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package lua5.1. 2024-09-22T00:58:05.675 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-22T00:58:05.688 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-22T00:58:05.702 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-22T00:58:05.722 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-22T00:58:05.742 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-22T00:58:06.020 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-22T00:58:06.025 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package lua-any. 2024-09-22T00:58:06.046 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-22T00:58:06.052 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-22T00:58:06.086 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-22T00:58:06.092 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-22T00:58:06.352 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package zip. 2024-09-22T00:58:06.368 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-22T00:58:06.411 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking zip (3.0-12build2) ... 2024-09-22T00:58:06.449 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua5.1. 2024-09-22T00:58:06.475 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-22T00:58:06.515 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-22T00:58:06.731 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package unzip. 2024-09-22T00:58:06.758 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-22T00:58:06.797 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-22T00:58:06.818 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-any. 2024-09-22T00:58:06.833 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-22T00:58:06.876 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-22T00:58:07.087 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package zip. 2024-09-22T00:58:07.114 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-22T00:58:07.143 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package luarocks. 2024-09-22T00:58:07.153 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking zip (3.0-12build2) ... 2024-09-22T00:58:07.169 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-22T00:58:07.201 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-22T00:58:07.482 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package unzip. 2024-09-22T00:58:07.508 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-22T00:58:07.548 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-22T00:58:07.936 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package luarocks. 2024-09-22T00:58:07.962 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-22T00:58:07.992 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package librgw2. 2024-09-22T00:58:08.002 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-22T00:58:08.020 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:08.092 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:08.385 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librgw2. 2024-09-22T00:58:08.411 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:08.451 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:08.653 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-rgw. 2024-09-22T00:58:08.680 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:08.719 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:09.004 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rgw. 2024-09-22T00:58:09.031 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:09.047 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-22T00:58:09.074 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-22T00:58:09.079 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:09.114 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-22T00:58:09.374 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-22T00:58:09.392 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libradosstriper1. 2024-09-22T00:58:09.401 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-22T00:58:09.418 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:09.440 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-22T00:58:09.458 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:09.777 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libradosstriper1. 2024-09-22T00:58:09.803 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:09.843 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:09.853 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-common. 2024-09-22T00:58:09.880 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:09.919 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:10.297 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-common. 2024-09-22T00:58:10.323 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:10.363 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:11.342 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-base. 2024-09-22T00:58:11.368 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:11.417 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:11.752 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-base. 2024-09-22T00:58:11.778 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:11.828 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:12.004 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-22T00:58:12.031 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-22T00:58:12.070 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-22T00:58:12.340 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cheroot. 2024-09-22T00:58:12.357 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-22T00:58:12.390 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-22T00:58:12.398 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-22T00:58:12.424 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-22T00:58:12.464 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-22T00:58:12.685 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-22T00:58:12.711 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-22T00:58:12.751 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-22T00:58:12.759 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cheroot. 2024-09-22T00:58:12.781 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-22T00:58:12.817 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-22T00:58:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-22T00:58:13.064 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-22T00:58:13.087 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-22T00:58:13.112 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-22T00:58:13.113 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-22T00:58:13.153 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-22T00:58:13.432 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-22T00:58:13.456 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-22T00:58:13.459 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-22T00:58:13.483 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-22T00:58:13.498 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-22T00:58:13.531 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-22T00:58:13.776 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-tempora. 2024-09-22T00:58:13.792 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-22T00:58:13.826 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-22T00:58:13.834 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-22T00:58:13.860 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-22T00:58:13.901 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-22T00:58:14.112 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-portend. 2024-09-22T00:58:14.139 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-22T00:58:14.178 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-22T00:58:14.195 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tempora. 2024-09-22T00:58:14.210 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-22T00:58:14.245 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-22T00:58:14.482 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-22T00:58:14.509 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-22T00:58:14.531 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-portend. 2024-09-22T00:58:14.548 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-22T00:58:14.558 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-22T00:58:14.598 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-22T00:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-22T00:58:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-22T00:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-22T00:58:14.903 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-22T00:58:14.917 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-22T00:58:14.942 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-22T00:58:15.204 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-22T00:58:15.230 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-22T00:58:15.270 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-22T00:58:15.271 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-natsort. 2024-09-22T00:58:15.298 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-22T00:58:15.337 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-22T00:58:15.616 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-natsort. 2024-09-22T00:58:15.642 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-22T00:58:15.657 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-logutils. 2024-09-22T00:58:15.682 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-22T00:58:15.684 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-22T00:58:15.724 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-22T00:58:16.027 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-22T00:58:16.053 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-22T00:58:16.085 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-22T00:58:16.169 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-logutils. 2024-09-22T00:58:16.195 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-22T00:58:16.227 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-22T00:58:16.379 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-mako. 2024-09-22T00:58:16.406 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-22T00:58:16.445 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-22T00:58:16.530 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-22T00:58:16.556 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-22T00:58:16.596 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-22T00:58:16.783 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-22T00:58:16.809 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-22T00:58:16.849 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-22T00:58:16.908 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-mako. 2024-09-22T00:58:16.934 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-22T00:58:16.974 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-22T00:58:17.152 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-22T00:58:17.179 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-22T00:58:17.218 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-22T00:58:17.311 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-22T00:58:17.337 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-22T00:58:17.377 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-22T00:58:17.521 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-webob. 2024-09-22T00:58:17.548 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-22T00:58:17.587 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-22T00:58:17.638 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-22T00:58:17.654 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-22T00:58:17.688 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-22T00:58:17.941 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-waitress. 2024-09-22T00:58:17.968 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-22T00:58:18.011 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-22T00:58:18.016 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webob. 2024-09-22T00:58:18.043 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-22T00:58:18.083 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-22T00:58:18.453 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-waitress. 2024-09-22T00:58:18.468 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-tempita. 2024-09-22T00:58:18.479 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-22T00:58:18.496 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-22T00:58:18.522 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-22T00:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-22T00:58:18.804 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-paste. 2024-09-22T00:58:18.830 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tempita. 2024-09-22T00:58:18.831 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-22T00:58:18.857 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-22T00:58:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-22T00:58:18.897 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-22T00:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-22T00:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-22T00:58:19.192 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-paste. 2024-09-22T00:58:19.216 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-22T00:58:19.218 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-22T00:58:19.250 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-22T00:58:19.519 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-22T00:58:19.545 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-22T00:58:19.570 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-22T00:58:19.585 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-22T00:58:19.596 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-22T00:58:19.636 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-22T00:58:19.905 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-webtest. 2024-09-22T00:58:19.932 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-22T00:58:19.971 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-22T00:58:19.998 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-22T00:58:20.024 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-22T00:58:20.072 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-22T00:58:20.299 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pecan. 2024-09-22T00:58:20.327 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-22T00:58:20.366 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-22T00:58:20.384 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webtest. 2024-09-22T00:58:20.411 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-22T00:58:20.451 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-22T00:58:20.745 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-certifi. 2024-09-22T00:58:20.762 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pecan. 2024-09-22T00:58:20.773 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-22T00:58:20.788 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-22T00:58:20.812 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-22T00:58:20.828 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-22T00:58:21.098 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-idna. 2024-09-22T00:58:21.125 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-22T00:58:21.173 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-22T00:58:21.200 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-certifi. 2024-09-22T00:58:21.227 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-22T00:58:21.266 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-22T00:58:21.484 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-urllib3. 2024-09-22T00:58:21.511 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-22T00:58:21.551 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-22T00:58:21.620 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-idna. 2024-09-22T00:58:21.646 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-22T00:58:21.694 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-22T00:58:21.871 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-requests. 2024-09-22T00:58:21.898 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-22T00:58:21.937 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-22T00:58:22.031 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-urllib3. 2024-09-22T00:58:22.057 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-22T00:58:22.097 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-22T00:58:22.273 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-22T00:58:22.301 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-22T00:58:22.348 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-22T00:58:22.426 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-requests. 2024-09-22T00:58:22.452 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-22T00:58:22.492 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-22T00:58:22.635 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-22T00:58:22.662 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:22.701 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:22.812 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-22T00:58:22.838 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-22T00:58:22.885 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-22T00:58:23.022 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-22T00:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:23.173 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-22T00:58:23.188 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:23.231 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:23.409 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-22T00:58:23.436 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-22T00:58:23.475 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-22T00:58:23.553 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-22T00:58:23.580 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:23.619 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:23.889 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-22T00:58:23.911 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-openssl. 2024-09-22T00:58:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-22T00:58:23.939 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-22T00:58:23.978 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-22T00:58:24.197 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-22T00:58:24.314 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr. 2024-09-22T00:58:24.342 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:24.380 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:24.767 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mon. 2024-09-22T00:58:24.794 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:24.833 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:25.084 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-openssl. 2024-09-22T00:58:25.111 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-22T00:58:25.150 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-22T00:58:25.437 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr. 2024-09-22T00:58:25.463 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:25.520 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:25.923 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mon. 2024-09-22T00:58:25.950 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:25.990 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:26.137 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-osd. 2024-09-22T00:58:26.165 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:26.204 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:26.819 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-osd. 2024-09-22T00:58:26.845 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:26.885 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:27.249 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph. 2024-09-22T00:58:27.264 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:27.298 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:27.585 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-fuse. 2024-09-22T00:58:27.612 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:27.651 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:27.921 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph. 2024-09-22T00:58:27.938 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:27.980 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:28.021 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mds. 2024-09-22T00:58:28.048 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:28.087 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:28.316 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-fuse. 2024-09-22T00:58:28.343 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:28.382 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:28.490 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package cephadm. 2024-09-22T00:58:28.517 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:28.557 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:28.761 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mds. 2024-09-22T00:58:28.787 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:28.826 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-22T00:58:28.827 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:28.842 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-22T00:58:28.876 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-22T00:58:29.205 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package cephadm. 2024-09-22T00:58:29.232 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:29.272 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:29.354 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-22T00:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-22T00:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-22T00:58:29.585 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-22T00:58:29.612 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-22T00:58:29.651 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-22T00:58:29.979 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-22T00:58:29.995 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-22T00:58:30.037 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-22T00:58:30.427 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-babel. 2024-09-22T00:58:30.454 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-22T00:58:30.493 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-22T00:58:30.830 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-jinja2. 2024-09-22T00:58:30.857 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-22T00:58:30.896 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-22T00:58:31.050 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-babel. 2024-09-22T00:58:31.078 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-22T00:58:31.117 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-22T00:58:31.241 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-22T00:58:31.268 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:31.307 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:31.437 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jinja2. 2024-09-22T00:58:31.464 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-22T00:58:31.503 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-22T00:58:31.652 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-22T00:58:31.679 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-22T00:58:31.718 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-22T00:58:31.848 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-22T00:58:31.875 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:31.914 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:31.971 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-routes. 2024-09-22T00:58:31.999 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-22T00:58:32.038 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-22T00:58:32.260 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-22T00:58:32.286 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-22T00:58:32.326 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-22T00:58:32.332 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-22T00:58:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:32.416 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:32.695 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-routes. 2024-09-22T00:58:32.722 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-22T00:58:32.770 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-22T00:58:33.115 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-22T00:58:33.142 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:33.207 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:33.563 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-22T00:58:33.590 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-22T00:58:33.629 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-22T00:58:34.366 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-joblib. 2024-09-22T00:58:34.393 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-22T00:58:34.432 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-22T00:58:34.504 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-22T00:58:34.519 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-22T00:58:34.553 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-22T00:58:34.777 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-22T00:58:34.805 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-22T00:58:34.835 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-22T00:58:35.015 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-joblib. 2024-09-22T00:58:35.031 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-22T00:58:35.065 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-22T00:58:35.113 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-sklearn. 2024-09-22T00:58:35.141 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-22T00:58:35.179 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-22T00:58:35.444 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-22T00:58:35.471 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-22T00:58:35.510 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-22T00:58:35.821 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sklearn. 2024-09-22T00:58:35.848 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-22T00:58:35.871 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-22T00:58:35.888 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-22T00:58:35.899 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:36.015 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:36.563 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-22T00:58:36.591 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:36.715 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:36.944 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-cachetools. 2024-09-22T00:58:36.971 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-22T00:58:37.010 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-22T00:58:37.313 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-22T00:58:37.340 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-22T00:58:37.379 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-22T00:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-22T00:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-22T00:58:37.774 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-22T00:58:37.910 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cachetools. 2024-09-22T00:58:37.926 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-22T00:58:37.962 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-22T00:58:38.086 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-rsa. 2024-09-22T00:58:38.114 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-22T00:58:38.152 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-22T00:58:38.205 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-22T00:58:38.221 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-22T00:58:38.255 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-22T00:58:38.465 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-google-auth. 2024-09-22T00:58:38.492 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-22T00:58:38.531 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-22T00:58:38.575 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-22T00:58:38.602 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-22T00:58:38.641 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-22T00:58:38.859 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-22T00:58:38.886 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-22T00:58:38.934 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-22T00:58:38.953 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rsa. 2024-09-22T00:58:38.980 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-22T00:58:39.020 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-22T00:58:39.262 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-websocket. 2024-09-22T00:58:39.290 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-22T00:58:39.328 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-22T00:58:39.331 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-google-auth. 2024-09-22T00:58:39.358 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-22T00:58:39.398 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-22T00:58:39.640 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-22T00:58:39.667 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-22T00:58:39.706 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-22T00:58:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-22T00:58:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-22T00:58:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-22T00:58:40.062 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-websocket. 2024-09-22T00:58:40.086 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-22T00:58:40.121 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-22T00:58:40.362 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-22T00:58:40.378 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:40.420 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:40.425 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-22T00:58:40.451 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-22T00:58:40.490 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-22T00:58:40.723 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-22T00:58:40.751 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-22T00:58:40.789 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-22T00:58:41.126 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-22T00:58:41.153 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-22T00:58:41.208 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-22T00:58:41.213 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-22T00:58:41.241 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:41.279 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:41.461 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package jq. 2024-09-22T00:58:41.478 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-22T00:58:41.519 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-22T00:58:41.599 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-22T00:58:41.626 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-22T00:58:41.665 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-22T00:58:41.772 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package socat. 2024-09-22T00:58:41.788 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-22T00:58:41.830 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-22T00:58:41.960 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-22T00:58:41.976 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-22T00:58:42.010 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-22T00:58:42.142 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package xmlstarlet. 2024-09-22T00:58:42.170 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-22T00:58:42.208 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-22T00:58:42.229 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package jq. 2024-09-22T00:58:42.245 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-22T00:58:42.288 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-22T00:58:42.540 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package socat. 2024-09-22T00:58:42.554 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-test. 2024-09-22T00:58:42.556 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-22T00:58:42.571 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:42.598 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-22T00:58:42.612 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:42.910 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package xmlstarlet. 2024-09-22T00:58:42.937 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-22T00:58:42.976 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-22T00:58:43.331 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-test. 2024-09-22T00:58:43.347 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:43.381 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:45.067 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package ceph-volume. 2024-09-22T00:58:45.094 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:45.133 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-22T00:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:45.503 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:45.585 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-volume. 2024-09-22T00:58:45.613 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T00:58:45.643 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:45.823 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-22T00:58:45.851 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-22T00:58:45.889 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-22T00:58:45.914 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-22T00:58:45.941 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:45.997 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:46.186 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-22T00:58:46.214 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-22T00:58:46.244 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-22T00:58:46.292 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-22T00:58:46.319 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-22T00:58:46.358 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-22T00:58:46.614 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package nvme-cli. 2024-09-22T00:58:46.642 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-22T00:58:46.646 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-22T00:58:46.674 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-22T00:58:46.681 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-22T00:58:46.704 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-22T00:58:47.075 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package nvme-cli. 2024-09-22T00:58:47.102 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-22T00:58:47.139 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package pkg-config. 2024-09-22T00:58:47.141 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-22T00:58:47.167 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-22T00:58:47.211 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-22T00:58:47.472 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-22T00:58:47.488 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-22T00:58:47.530 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-22T00:58:47.564 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package pkg-config. 2024-09-22T00:58:47.591 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-22T00:58:47.630 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-22T00:58:47.883 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-22T00:58:47.910 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-22T00:58:47.941 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-22T00:58:47.949 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-22T00:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-22T00:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-22T00:58:48.277 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pastescript. 2024-09-22T00:58:48.294 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-22T00:58:48.327 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-22T00:58:48.606 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pluggy. 2024-09-22T00:58:48.611 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-22T00:58:48.634 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-22T00:58:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-22T00:58:48.673 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-22T00:58:48.677 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-22T00:58:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pastescript. 2024-09-22T00:58:48.974 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-22T00:58:48.983 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-psutil. 2024-09-22T00:58:49.005 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-22T00:58:49.011 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-22T00:58:49.050 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-22T00:58:49.250 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pluggy. 2024-09-22T00:58:49.266 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-22T00:58:49.300 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-22T00:58:49.386 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-py. 2024-09-22T00:58:49.414 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-22T00:58:49.452 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-22T00:58:49.603 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-psutil. 2024-09-22T00:58:49.631 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-22T00:58:49.669 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-22T00:58:49.781 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pygments. 2024-09-22T00:58:49.809 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-22T00:58:49.847 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-22T00:58:50.023 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-py. 2024-09-22T00:58:50.050 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-22T00:58:50.089 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-22T00:58:50.276 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-22T00:58:50.304 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-22T00:58:50.351 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-22T00:58:50.376 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pygments. 2024-09-22T00:58:50.403 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-22T00:58:50.442 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-22T00:58:50.646 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-toml. 2024-09-22T00:58:50.673 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-22T00:58:50.712 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-22T00:58:50.854 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-22T00:58:50.871 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-22T00:58:50.913 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-22T00:58:50.998 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-pytest. 2024-09-22T00:58:51.026 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-22T00:58:51.064 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-22T00:58:51.215 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-toml. 2024-09-22T00:58:51.243 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-22T00:58:51.282 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-22T00:58:51.418 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package python3-simplejson. 2024-09-22T00:58:51.446 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-22T00:58:51.509 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-22T00:58:51.585 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pytest. 2024-09-22T00:58:51.612 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-22T00:58:51.651 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-22T00:58:51.795 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-22T00:58:51.823 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-22T00:58:51.861 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-22T00:58:51.988 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-simplejson. 2024-09-22T00:58:52.016 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-22T00:58:52.055 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-22T00:58:52.324 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-22T00:58:52.340 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-22T00:58:52.399 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-22T00:58:52.532 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package radosgw. 2024-09-22T00:58:52.560 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:52.598 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:53.137 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package radosgw. 2024-09-22T00:58:53.164 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:53.203 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:53.427 INFO:teuthology.orchestra.run.smithi002.stdout:Selecting previously unselected package rbd-fuse. 2024-09-22T00:58:53.455 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:53.493 INFO:teuthology.orchestra.run.smithi002.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:53.783 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-22T00:58:54.014 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package rbd-fuse. 2024-09-22T00:58:54.031 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T00:58:54.072 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:54.081 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-22T00:58:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-22T00:58:54.377 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-22T00:58:54.437 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-22T00:58:54.628 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-22T00:58:54.767 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-22T00:58:54.770 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-22T00:58:55.020 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-22T00:58:55.185 INFO:teuthology.orchestra.run.smithi002.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-22T00:58:55.326 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:55.345 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-22T00:58:55.795 INFO:teuthology.orchestra.run.smithi119.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-22T00:58:55.865 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:55.885 INFO:teuthology.orchestra.run.smithi002.stdout:Adding system user cephadm....done 2024-09-22T00:58:56.040 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-22T00:58:56.300 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-22T00:58:56.559 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-22T00:58:56.685 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-22T00:58:56.945 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-22T00:58:56.990 INFO:teuthology.orchestra.run.smithi119.stdout:Adding system user cephadm....done 2024-09-22T00:58:57.210 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-22T00:58:57.212 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-22T00:58:57.339 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-22T00:58:57.597 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-22T00:58:57.632 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-22T00:58:57.934 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-22T00:58:58.002 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-22T00:58:58.061 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-22T00:58:58.269 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-22T00:58:58.320 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-22T00:58:58.570 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-22T00:58:58.580 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-22T00:58:58.681 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-22T00:58:58.830 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-22T00:58:58.966 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-22T00:58:59.089 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-22T00:58:59.318 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-22T00:58:59.348 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-22T00:58:59.577 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-22T00:58:59.591 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:58:59.842 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-22T00:58:59.862 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-22T00:58:59.968 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-22T00:59:00.094 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-22T00:59:00.138 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-22T00:59:00.236 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-22T00:59:00.389 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-22T00:59:00.496 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-22T00:59:00.624 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-22T00:59:00.867 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:01.013 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-22T00:59:01.044 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-22T00:59:01.107 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-22T00:59:01.126 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-22T00:59:01.225 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-22T00:59:01.252 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-22T00:59:01.378 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-22T00:59:01.504 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-22T00:59:01.593 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-22T00:59:01.719 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-22T00:59:01.755 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-22T00:59:01.995 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-22T00:59:02.109 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-22T00:59:02.146 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-22T00:59:02.208 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-22T00:59:02.262 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-22T00:59:02.334 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-22T00:59:02.572 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up zip (3.0-12build2) ... 2024-09-22T00:59:02.673 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-22T00:59:02.702 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-22T00:59:02.828 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-22T00:59:03.088 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-22T00:59:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-22T00:59:03.331 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-22T00:59:03.576 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-22T00:59:03.632 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up zip (3.0-12build2) ... 2024-09-22T00:59:03.702 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-22T00:59:03.758 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-22T00:59:03.828 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-22T00:59:04.129 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-22T00:59:04.410 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-22T00:59:04.564 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-22T00:59:04.669 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-22T00:59:04.795 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-22T00:59:04.814 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-22T00:59:04.913 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-22T00:59:05.191 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-22T00:59:05.197 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-22T00:59:05.484 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-22T00:59:05.573 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-22T00:59:05.816 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-22T00:59:05.844 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-22T00:59:06.103 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-22T00:59:06.193 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-22T00:59:06.229 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-22T00:59:06.547 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-22T00:59:06.812 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-22T00:59:06.865 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-22T00:59:07.047 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-22T00:59:07.173 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:07.457 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-22T00:59:07.792 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-22T00:59:08.120 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-22T00:59:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:59:08.531 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-22T00:59:08.799 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-22T00:59:08.889 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-22T00:59:08.958 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-22T00:59:09.084 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-22T00:59:09.149 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:59:09.283 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-22T00:59:09.394 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-22T00:59:09.551 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-22T00:59:09.653 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-22T00:59:09.686 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-22T00:59:09.812 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-22T00:59:09.913 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:59:10.039 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-22T00:59:10.113 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-22T00:59:10.349 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-22T00:59:10.355 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-22T00:59:10.590 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:59:10.600 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-22T00:59:10.716 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-22T00:59:10.884 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-22T00:59:11.017 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-22T00:59:11.143 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-22T00:59:11.277 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-22T00:59:11.395 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-22T00:59:11.520 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-22T00:59:11.671 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-22T00:59:11.796 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-22T00:59:11.796 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-22T00:59:12.039 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-22T00:59:12.081 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-22T00:59:12.182 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-22T00:59:12.298 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-22T00:59:12.424 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-22T00:59:12.466 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-22T00:59:12.700 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-22T00:59:12.784 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-22T00:59:12.826 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-22T00:59:13.052 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-22T00:59:13.114 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-22T00:59:13.178 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-22T00:59:13.329 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-22T00:59:13.406 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-22T00:59:13.419 INFO:teuthology.orchestra.run.smithi002.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-22T00:59:13.640 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-22T00:59:13.664 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-22T00:59:13.766 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-22T00:59:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-22T00:59:13.892 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-22T00:59:13.982 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-22T00:59:14.174 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-22T00:59:14.235 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-22T00:59:14.361 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-22T00:59:14.442 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-22T00:59:14.568 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-22T00:59:14.746 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-22T00:59:14.828 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:59:14.953 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-22T00:59:14.997 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-22T00:59:15.107 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-22T00:59:15.296 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-22T00:59:15.366 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T00:59:15.517 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-22T00:59:15.589 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-22T00:59:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-22T00:59:15.865 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-22T00:59:15.974 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-22T00:59:16.119 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-22T00:59:16.250 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-22T00:59:16.387 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-22T00:59:16.513 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-22T00:59:16.529 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-22T00:59:16.789 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-22T00:59:16.873 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-22T00:59:17.057 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-22T00:59:17.258 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-22T00:59:17.418 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-22T00:59:17.543 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-22T00:59:17.787 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-22T00:59:17.904 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-22T00:59:18.022 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:18.047 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-22T00:59:18.123 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:18.232 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-22T00:59:18.374 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-22T00:59:18.500 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:18.626 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:18.677 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:18.752 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-22T00:59:18.804 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-22T00:59:19.089 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:19.215 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:19.332 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:19.522 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:19.533 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-22T00:59:19.533 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-22T00:59:19.631 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:19.758 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-22T00:59:20.009 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:20.034 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:20.134 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:20.144 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:20.260 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:20.387 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-22T00:59:20.485 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-22T00:59:20.486 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-22T00:59:20.929 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:21.072 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:21.198 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:21.324 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-22T00:59:21.707 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:21.808 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:21.925 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:22.068 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:22.211 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:22.577 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:22.704 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:22.753 INFO:teuthology.orchestra.run.smithi002.stdout:Adding group ceph....done 2024-09-22T00:59:22.805 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:22.906 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:23.130 INFO:teuthology.orchestra.run.smithi002.stdout:Adding system user ceph....done 2024-09-22T00:59:23.298 INFO:teuthology.orchestra.run.smithi002.stdout:Setting system user ceph properties....done 2024-09-22T00:59:23.316 INFO:teuthology.orchestra.run.smithi002.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-22T00:59:23.467 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-22T00:59:23.583 INFO:teuthology.orchestra.run.smithi119.stdout:Adding group ceph....done 2024-09-22T00:59:23.769 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-22T00:59:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:Adding system user ceph....done 2024-09-22T00:59:24.111 INFO:teuthology.orchestra.run.smithi119.stdout:Setting system user ceph properties....done 2024-09-22T00:59:24.125 INFO:teuthology.orchestra.run.smithi119.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-22T00:59:24.220 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:24.265 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-22T00:59:24.346 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:24.564 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-22T00:59:24.793 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-22T00:59:24.794 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-22T00:59:25.034 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:25.151 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:25.239 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:25.586 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-22T00:59:25.586 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-22T00:59:25.736 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-22T00:59:26.144 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:26.196 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:26.494 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-22T00:59:26.505 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-22T00:59:26.505 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-22T00:59:26.963 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:27.102 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:27.203 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-22T00:59:27.203 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-22T00:59:27.334 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-22T00:59:27.334 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-22T00:59:27.698 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:27.812 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:27.922 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-22T00:59:27.922 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-22T00:59:28.076 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-22T00:59:28.076 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-22T00:59:28.374 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:28.577 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:28.648 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-22T00:59:28.648 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-22T00:59:28.776 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:29.035 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:29.092 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:29.193 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:29.319 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:29.347 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-22T00:59:29.347 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-22T00:59:29.531 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-22T00:59:29.531 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-22T00:59:29.818 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:29.963 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:29.969 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:30.089 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:30.095 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:30.246 INFO:teuthology.orchestra.run.smithi002.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T00:59:30.607 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-22T00:59:30.733 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-22T00:59:30.760 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-22T00:59:30.870 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-22T00:59:30.886 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-22T00:59:31.049 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-22T00:59:31.678 INFO:teuthology.orchestra.run.smithi002.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-22T00:59:32.073 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-22T00:59:32.330 INFO:teuthology.orchestra.run.smithi002.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-22T00:59:32.335 DEBUG:teuthology.parallel:result is None 2024-09-22T00:59:32.700 INFO:teuthology.orchestra.run.smithi119.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-22T00:59:32.704 DEBUG:teuthology.parallel:result is None 2024-09-22T00:59:32.705 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T00:59:32.841 DEBUG:teuthology.orchestra.run.smithi002:> dpkg-query -W -f '${Version}' ceph 2024-09-22T00:59:32.861 INFO:teuthology.orchestra.run.smithi002.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:59:32.861 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:59:32.861 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-22T00:59:32.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T00:59:33.001 DEBUG:teuthology.orchestra.run.smithi119:> dpkg-query -W -f '${Version}' ceph 2024-09-22T00:59:33.020 INFO:teuthology.orchestra.run.smithi119.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:59:33.021 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T00:59:33.021 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-22T00:59:33.022 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-22T00:59:33.022 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:33.023 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T00:59:33.037 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:59:33.037 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T00:59:33.077 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-22T00:59:33.077 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:33.078 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T00:59:33.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T00:59:33.146 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:59:33.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T00:59:33.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T00:59:33.212 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-22T00:59:33.212 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:33.212 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T00:59:33.226 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T00:59:33.282 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:59:33.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T00:59:33.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T00:59:33.352 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-22T00:59:33.352 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:33.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T00:59:33.365 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T00:59:33.418 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:59:33.419 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T00:59:33.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T00:59:33.484 INFO:teuthology.run_tasks:Running task ceph... 2024-09-22T00:59:33.552 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-22T00:59:33.552 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /var/log/ceph 2024-09-22T00:59:33.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2024-09-22T00:59:33.567 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-22T00:59:33.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T00:59:33.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T00:59:33.620 INFO:tasks.ceph:Creating extra log directories... 2024-09-22T00:59:33.620 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T00:59:33.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T00:59:33.686 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-22T00:59:33.686 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-22T00:59:33.687 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914593', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1478, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '24/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi002.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVucGqkrSf/q5/xLYTxKBUYe8McKkxnlpURmHbMl6RSj4t4cUlqh8UsTs6AGqz+GLdBnQwHwYd7S03wyJyBegc=', 'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBuhwlnBFlT6NiBF5l3x/0ibeumHbhKYQHKpQttpi4ao6q6B9ef3LEtw+5y26kkALWNOnIwTAhf2dln/30dpbTU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-22T00:59:33.687 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T00:59:33.719 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T00:59:33.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T00:59:33.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T00:59:33.781 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:33.781 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T00:59:33.826 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T00:59:33.827 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2024-09-22T00:59:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T00:59:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T00:59:33.875 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-22T00:59:33.875 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T00:59:33.875 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-22 00:54:04.774570671 +0000 2024-09-22T00:59:33.875 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-22 00:54:04.770570707 +0000 2024-09-22T00:59:33.875 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-22 00:54:04.770570707 +0000 2024-09-22T00:59:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2024-09-22T00:59:33.875 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T00:59:33.929 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-22T00:59:33.929 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-22T00:59:33.929 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000282037 s, 1.8 MB/s 2024-09-22T00:59:33.930 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T00:59:33.980 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2024-09-22T00:59:34.026 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T00:59:34.027 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T00:59:34.027 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-09-22T00:59:34.027 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T00:59:34.027 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-22 00:54:05.486564312 +0000 2024-09-22T00:59:34.027 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-22 00:54:05.214566741 +0000 2024-09-22T00:59:34.027 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-22 00:54:05.214566741 +0000 2024-09-22T00:59:34.027 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2024-09-22T00:59:34.027 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T00:59:34.080 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-22T00:59:34.080 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-22T00:59:34.080 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000202599 s, 2.5 MB/s 2024-09-22T00:59:34.081 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T00:59:34.132 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2024-09-22T00:59:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T00:59:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T00:59:34.179 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-09-22T00:59:34.179 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T00:59:34.179 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-22 00:54:06.010559632 +0000 2024-09-22T00:59:34.179 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-22 00:54:05.714562275 +0000 2024-09-22T00:59:34.179 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-22 00:54:05.714562275 +0000 2024-09-22T00:59:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2024-09-22T00:59:34.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T00:59:34.232 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-22T00:59:34.233 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-22T00:59:34.233 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000198126 s, 2.6 MB/s 2024-09-22T00:59:34.234 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T00:59:34.284 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2024-09-22T00:59:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T00:59:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T00:59:34.331 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-09-22T00:59:34.331 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T00:59:34.331 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-22 00:54:06.494555308 +0000 2024-09-22T00:59:34.331 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-22 00:54:06.230557666 +0000 2024-09-22T00:59:34.331 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-22 00:54:06.230557666 +0000 2024-09-22T00:59:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2024-09-22T00:59:34.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T00:59:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-22T00:59:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-22T00:59:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000306614 s, 1.7 MB/s 2024-09-22T00:59:34.387 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T00:59:34.436 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:34.436 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:59:34.436 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T00:59:34.442 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T00:59:34.442 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-09-22T00:59:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T00:59:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T00:59:34.490 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-09-22T00:59:34.490 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T00:59:34.490 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-22 00:54:01.906206526 +0000 2024-09-22T00:59:34.490 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-22 00:54:01.686208491 +0000 2024-09-22T00:59:34.490 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-22 00:54:01.686208491 +0000 2024-09-22T00:59:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-09-22T00:59:34.490 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T00:59:34.544 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-22T00:59:34.544 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-22T00:59:34.544 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000451324 s, 1.1 MB/s 2024-09-22T00:59:34.545 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T00:59:34.595 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-09-22T00:59:34.641 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T00:59:34.642 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T00:59:34.642 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-09-22T00:59:34.642 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T00:59:34.642 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-22 00:54:02.394202167 +0000 2024-09-22T00:59:34.642 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-22 00:54:02.146204382 +0000 2024-09-22T00:59:34.642 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-22 00:54:02.146204382 +0000 2024-09-22T00:59:34.642 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-09-22T00:59:34.642 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T00:59:34.695 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-22T00:59:34.696 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-22T00:59:34.696 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00043611 s, 1.2 MB/s 2024-09-22T00:59:34.697 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T00:59:34.746 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-09-22T00:59:34.798 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T00:59:34.798 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T00:59:34.798 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-09-22T00:59:34.798 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T00:59:34.798 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-22 00:54:02.898197665 +0000 2024-09-22T00:59:34.798 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-22 00:54:02.650199881 +0000 2024-09-22T00:59:34.799 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-22 00:54:02.650199881 +0000 2024-09-22T00:59:34.799 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-09-22T00:59:34.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T00:59:34.853 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-22T00:59:34.853 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-22T00:59:34.853 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000430823 s, 1.2 MB/s 2024-09-22T00:59:34.855 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T00:59:34.902 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-09-22T00:59:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T00:59:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T00:59:34.949 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-09-22T00:59:34.949 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T00:59:34.949 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-22 00:54:03.394193235 +0000 2024-09-22T00:59:34.950 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-22 00:54:03.126195628 +0000 2024-09-22T00:59:34.950 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-22 00:54:03.126195628 +0000 2024-09-22T00:59:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-09-22T00:59:34.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T00:59:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-22T00:59:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-22T00:59:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000385531 s, 1.3 MB/s 2024-09-22T00:59:35.005 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T00:59:35.054 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:35.055 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-22T00:59:35.055 INFO:tasks.ceph:Generating config... 2024-09-22T00:59:35.056 INFO:tasks.ceph:[client] rbd cache = True 2024-09-22T00:59:35.056 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-09-22T00:59:35.056 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-22T00:59:35.056 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-22T00:59:35.056 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-22T00:59:35.056 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-22T00:59:35.056 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-22T00:59:35.056 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-22T00:59:35.056 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-22T00:59:35.056 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-22T00:59:35.056 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-22T00:59:35.056 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-22T00:59:35.057 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-22T00:59:35.058 INFO:tasks.ceph:Setting up mon.a... 2024-09-22T00:59:35.058 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-22T00:59:35.094 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.keyring 2024-09-22T00:59:35.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-22T00:59:35.187 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T00:59:35.242 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.2'), ('mon.c', '[v2:172.21.15.2:3301,v1:172.21.15.2:6790]'), ('mon.b', '172.21.15.119')] 2024-09-22T00:59:35.243 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.2,[v2:172.21.15.2:3301,v1:172.21.15.2:6790],172.21.15.119', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-22T00:59:35.243 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:35.243 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T00:59:35.290 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.2 --addv c '[v2:172.21.15.2:3301,v1:172.21.15.2:6790]' --add b 172.21.15.119 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: generated fsid 7d08b428-4185-4f0f-a0fc-f7dc1fefdce9 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = quincy 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 0 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 7d08b428-4185-4f0f-a0fc-f7dc1fefdce9 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:last_changed 2024-09-22T00:59:35.378673+0000 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-09-22T00:59:35.378673+0000 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:min_mon_release 17 (quincy) 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:election_strategy: 1 2024-09-22T00:59:35.378 INFO:teuthology.orchestra.run.smithi002.stdout:0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-22T00:59:35.379 INFO:teuthology.orchestra.run.smithi002.stdout:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-22T00:59:35.379 INFO:teuthology.orchestra.run.smithi002.stdout:2: [v2:172.21.15.2:3301/0,v1:172.21.15.2:6790/0] mon.c 2024-09-22T00:59:35.379 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-22T00:59:35.380 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T00:59:35.384 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7d08b428-4185-4f0f-a0fc-f7dc1fefdce9... 2024-09-22T00:59:35.385 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T00:59:35.427 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T00:59:35.456 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2024-09-22T00:59:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: chdir = "" 2024-09-22T00:59:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T00:59:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: auth supported = cephx 2024-09-22T00:59:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: filestore xattr use omap = true 2024-09-22T00:59:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: mon clock drift allowed = 1.000 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: osd crush chooseleaf type = 0 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: auth debug = true 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on old message = true 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on bug = true 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: mon pg warn max object skew = 0 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default size = 2 2024-09-22T00:59:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow primary affinity = true 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow pg remap = true 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on legacy crush tunables = false 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on crush straw calc version zero = false 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on no sortbitwise = false 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on osd down out interval zero = false 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on too few osds = false 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: mon_allow_pool_size_one = true 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: osd default data pool replay window = 5 2024-09-22T00:59:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: mon allow pool delete = true 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: mon cluster log file level = debug 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: debug asserts on shutdown = true 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: mon health detail to clog = false 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: mon host = "172.21.15.2,[v2:172.21.15.2:3301,v1:172.21.15.2:6790],172.21.15.119" 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: mon client directed command retry = 5 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on pool no app = False 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: ms inject socket failures = 5000 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = 7d08b428-4185-4f0f-a0fc-f7dc1fefdce9 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout:[osd] 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: osd journal size = 100 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub load threshold = 5.0 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub max interval = 600 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd mclock profile = high_recovery_ops 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd recover clone overlap = true 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd recovery max chunk = 1048576 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug shutdown = true 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug op order = true 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug verify stray on activate = true 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug trim objects = true 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd open classes on start = true 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug pg log writeout = true 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: osd deep scrub update digest min age = 30 2024-09-22T00:59:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: osd map max advance = 10 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: journal zero on create = true 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: filestore ondisk finisher threads = 3 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: filestore apply finisher threads = 3 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: bdev debug aio = true 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug misdirected ops = true 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: bdev async discard = True 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: bdev enable discard = True 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore block size = 96636764160 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore fsck on mount = True 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluefs = 1/20 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluestore = 1/20 2024-09-22T00:59:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: debug osd = 20 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: debug rocksdb = 4/10 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd backfillfull_ratio = 0.85 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd full ratio = 0.9 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd nearfull ratio = 0.8 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: osd blocked scrub grace period = 3600 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: osd failsafe full ratio = 0.95 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: osd max markdown count = 1000 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: osd memory target = 1610612736 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: osd objectstore = bluestore 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: osd shutdown pgref assert = True 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.462 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr] 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: debug mgr = 20 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: mgr/telemetry/nag = false 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout:[mon] 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: debug paxos = 20 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: mon data avail warn = 5 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: mon mgr mkfs grace = 240 2024-09-22T00:59:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd reporter subtree level = osd 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd prime pg temp = true 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: # 1m isn't quite enough 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: mon_down_mkfs_grace = 2m 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_filestore_osds = false 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout:[client] 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: rgw cache enabled = true 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable ops log = true 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable usage log = true 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: rbd cache = True 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: rbd cache policy = writearound 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.a] 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.c] 2024-09-22T00:59:35.465 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.b] 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-09-22T00:59:35.466 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-09-22T00:59:35.467 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.2,[v2:172.21.15.2:3301,v1:172.21.15.2:6790],172.21.15.119" 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon client directed command retry = 5 2024-09-22T00:59:35.468 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on pool no app = False 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: ms inject socket failures = 5000 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = 7d08b428-4185-4f0f-a0fc-f7dc1fefdce9 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-09-22T00:59:35.469 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.470 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: osd blocked scrub grace period = 3600 2024-09-22T00:59:35.471 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: osd max markdown count = 1000 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: osd memory target = 1610612736 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: osd shutdown pgref assert = True 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.472 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T00:59:35.473 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: rbd cache = True 2024-09-22T00:59:35.474 INFO:teuthology.orchestra.run.smithi119.stdout: rbd cache policy = writearound 2024-09-22T00:59:35.475 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-09-22T00:59:35.475 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-09-22T00:59:35.475 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-09-22T00:59:35.476 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-22T00:59:35.476 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-22T00:59:35.567 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-22T00:59:35.567 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:35.567 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-22T00:59:35.572 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:35.572 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-22T00:59:35.618 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T00:59:35.618 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:35.618 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T00:59:35.619 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T00:59:35.683 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T00:59:35.683 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:35.731 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T00:59:35.731 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:59:35.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T00:59:35.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T00:59:35.754 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T00:59:35.754 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:35.802 INFO:tasks.ceph:Setting up mon nodes... 2024-09-22T00:59:35.802 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-22T00:59:35.802 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-22T00:59:35.845 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-22T00:59:35.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T00:59:35.890 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T00:59:35.904 INFO:tasks.ceph:Setting up mds nodes... 2024-09-22T00:59:35.904 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-22T00:59:35.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-22T00:59:35.940 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-22T00:59:35.954 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-22T00:59:35.987 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-22T00:59:35.998 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-22T00:59:35.998 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-22T00:59:35.998 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-22T00:59:36.012 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:36.012 INFO:tasks.ceph:role: osd.0 2024-09-22T00:59:36.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T00:59:36.012 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T00:59:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T00:59:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-22T00:59:36.092 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-22T00:59:36.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-22T00:59:36.151 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-22T00:59:36.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:59:36.203 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T00:59:36.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-22T00:59:36.259 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:36.259 INFO:tasks.ceph:role: osd.1 2024-09-22T00:59:36.259 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T00:59:36.259 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T00:59:36.313 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T00:59:36.314 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T00:59:36.337 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-22T00:59:36.339 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-22T00:59:36.340 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-22T00:59:36.397 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-22T00:59:36.450 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T00:59:36.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:59:36.451 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-22T00:59:36.508 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:36.508 INFO:tasks.ceph:role: osd.2 2024-09-22T00:59:36.508 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T00:59:36.509 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T00:59:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T00:59:36.564 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T00:59:36.587 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-22T00:59:36.588 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-22T00:59:36.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-22T00:59:36.649 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-22T00:59:36.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:59:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T00:59:36.703 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-22T00:59:36.760 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:36.760 INFO:tasks.ceph:role: osd.3 2024-09-22T00:59:36.760 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T00:59:36.760 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T00:59:36.814 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T00:59:36.838 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-22T00:59:36.839 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-22T00:59:36.840 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-22T00:59:36.897 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-22T00:59:36.951 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T00:59:36.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:59:36.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:37.053+0000 7f55f63c67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-22T00:59:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:37.053+0000 7f55f63c67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-22T00:59:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:37.053+0000 7f55f63c67c0 -1 bdev(0x55aea19d5800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-22T00:59:37.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:37.053+0000 7f55f63c67c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-22T00:59:40.460 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-22T00:59:40.474 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:40.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:40.557+0000 7fcf2d43a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-22T00:59:40.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:40.557+0000 7fcf2d43a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-22T00:59:40.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:40.557+0000 7fcf2d43a7c0 -1 bdev(0x55de572cf800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-22T00:59:40.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:40.557+0000 7fcf2d43a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-22T00:59:44.664 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-22T00:59:44.718 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:44.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:44.805+0000 7f726c84f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-22T00:59:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:44.805+0000 7f726c84f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-22T00:59:44.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:44.805+0000 7f726c84f7c0 -1 bdev(0x556044f19800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-22T00:59:44.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:44.805+0000 7f726c84f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-22T00:59:48.649 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-22T00:59:48.711 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:48.797+0000 7ff022f7b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-22T00:59:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:48.797+0000 7ff022f7b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-22T00:59:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:48.797+0000 7ff022f7b7c0 -1 bdev(0x555e7a329800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-22T00:59:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T00:59:48.797+0000 7ff022f7b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-22T00:59:52.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-22T00:59:52.722 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-22T00:59:52.737 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:52.737 INFO:tasks.ceph:role: osd.4 2024-09-22T00:59:52.737 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T00:59:52.737 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T00:59:52.800 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T00:59:52.800 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T00:59:52.800 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T00:59:52.800 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T00:59:52.801 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T00:59:52.801 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-22T00:59:52.801 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T00:59:52.801 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T00:59:52.801 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T00:59:52.801 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T00:59:53.257 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-22T00:59:53.258 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-22T00:59:53.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-22T00:59:53.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-22T00:59:53.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:59:53.359 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T00:59:53.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-22T00:59:53.410 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:53.410 INFO:tasks.ceph:role: osd.5 2024-09-22T00:59:53.410 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T00:59:53.410 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T00:59:53.472 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T00:59:53.472 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T00:59:53.472 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T00:59:53.472 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T00:59:53.473 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T00:59:53.473 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-22T00:59:53.473 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T00:59:53.473 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T00:59:53.473 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T00:59:53.473 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T00:59:53.938 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-22T00:59:53.941 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-22T00:59:53.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-22T00:59:54.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-22T00:59:54.102 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T00:59:54.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:59:54.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-22T00:59:54.156 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:54.156 INFO:tasks.ceph:role: osd.6 2024-09-22T00:59:54.156 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T00:59:54.156 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-22T00:59:54.221 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T00:59:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T00:59:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T00:59:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T00:59:54.221 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T00:59:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-22T00:59:54.221 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T00:59:54.221 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T00:59:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T00:59:54.222 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T00:59:54.699 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-22T00:59:54.701 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-22T00:59:54.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-22T00:59:54.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-22T00:59:54.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:59:54.855 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T00:59:54.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-22T00:59:54.909 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T00:59:54.909 INFO:tasks.ceph:role: osd.7 2024-09-22T00:59:54.909 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T00:59:54.909 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T00:59:54.973 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T00:59:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-22T00:59:55.410 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-22T00:59:55.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-22T00:59:55.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-22T00:59:55.518 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T00:59:55.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T00:59:55.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:55.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T00:59:55.630+0000 7f46c7c6f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-22T00:59:55.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T00:59:55.634+0000 7f46c7c6f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-22T00:59:55.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T00:59:55.634+0000 7f46c7c6f7c0 -1 bdev(0x55f82931d800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-22T00:59:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T00:59:55.634+0000 7f46c7c6f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-22T00:59:59.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-22T00:59:59.521 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T00:59:59.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T00:59:59.598+0000 7f7b3bf5b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-22T00:59:59.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T00:59:59.598+0000 7f7b3bf5b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-22T00:59:59.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T00:59:59.598+0000 7f7b3bf5b7c0 -1 bdev(0x558b603c1800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-22T00:59:59.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T00:59:59.598+0000 7f7b3bf5b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-22T01:00:03.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-22T01:00:03.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:00:03.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T01:00:03.366+0000 7f2349ac67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-22T01:00:03.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T01:00:03.366+0000 7f2349ac67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-22T01:00:03.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T01:00:03.366+0000 7f2349ac67c0 -1 bdev(0x5586fa771800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-22T01:00:03.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T01:00:03.366+0000 7f2349ac67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-22T01:00:07.020 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-22T01:00:07.081 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:00:07.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T01:00:07.158+0000 7f1468f727c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-22T01:00:07.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T01:00:07.158+0000 7f1468f727c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-22T01:00:07.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T01:00:07.158+0000 7f1468f727c0 -1 bdev(0x55da1b005800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-22T01:00:07.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-22T01:00:07.158+0000 7f1468f727c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-22T01:00:10.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-22T01:00:10.585 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-22T01:00:10.586 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:10.586 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-22T01:00:10.601 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:10.601 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-22T01:00:10.654 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:10.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-22T01:00:10.709 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:10.709 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-22T01:00:10.765 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:10.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-22T01:00:10.821 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:10.821 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-22T01:00:10.834 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:10.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-22T01:00:10.887 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:10.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-22T01:00:10.939 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:10.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-22T01:00:10.991 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:10.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-22T01:00:11.043 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:11.043 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-22T01:00:11.049 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:11.049 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-22T01:00:11.089 INFO:tasks.ceph:Adding keys to all mons... 2024-09-22T01:00:11.089 DEBUG:teuthology.orchestra.run.smithi002:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-22T01:00:11.095 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-22T01:00:11.135 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.y] 2024-09-22T01:00:11.135 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD3a+9mIS5sMhAA6EuIR0diEA1rbqRuQ213Rw== 2024-09-22T01:00:11.135 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.0] 2024-09-22T01:00:11.135 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD5a+9mM/lUAxAAT/2ooLRTxruNIBzIVdCReg== 2024-09-22T01:00:11.135 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.1] 2024-09-22T01:00:11.135 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD8a+9m9zNNIRAAc6BJl69Ms6kH47b+AU4Rkw== 2024-09-22T01:00:11.135 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.2] 2024-09-22T01:00:11.135 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQAAbO9mCF8bMBAAjO/2I1we00jbSbl/pcdYXA== 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.3] 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQAEbO9mtB6cLxAACp+kDKymjjmBmQJIUI4LaQ== 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.x] 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD3a+9myrQbNRAASCM3pb9rlr5i+dsSJny9aA== 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.4] 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQALbO9mWFnIJRAAG6xMr5eLMQlKbVL96tTSjQ== 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.5] 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQAPbO9mbZ/oIxAAPraX61MQajE4wzVMBPbycw== 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.6] 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQATbO9mOCUNFhAA5vZuwCoPwjAke0toDcCm8w== 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.7] 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQAXbO9m7pvKCRAANI4dTm8PkKobIXiADv9LcA== 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2024-09-22T01:00:11.136 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD3a+9mz5oWOBAA7lhxo8ng8NFLCb7R+jE6MQ== 2024-09-22T01:00:11.137 INFO:teuthology.orchestra.run.smithi002.stdout:[client.1] 2024-09-22T01:00:11.137 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD3a+9mFXznOhAAF3Qq1qWduxaK+OO8TSH5bA== 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQD3a+9mIS5sMhAA6EuIR0diEA1rbqRuQ213Rw== 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQD5a+9mM/lUAxAAT/2ooLRTxruNIBzIVdCReg== 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQD8a+9m9zNNIRAAc6BJl69Ms6kH47b+AU4Rkw== 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAAbO9mCF8bMBAAjO/2I1we00jbSbl/pcdYXA== 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2024-09-22T01:00:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAEbO9mtB6cLxAACp+kDKymjjmBmQJIUI4LaQ== 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQD3a+9myrQbNRAASCM3pb9rlr5i+dsSJny9aA== 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQALbO9mWFnIJRAAG6xMr5eLMQlKbVL96tTSjQ== 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAPbO9mbZ/oIxAAPraX61MQajE4wzVMBPbycw== 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQATbO9mOCUNFhAA5vZuwCoPwjAke0toDcCm8w== 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2024-09-22T01:00:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAXbO9m7pvKCRAANI4dTm8PkKobIXiADv9LcA== 2024-09-22T01:00:11.145 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-09-22T01:00:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQD3a+9mz5oWOBAA7lhxo8ng8NFLCb7R+jE6MQ== 2024-09-22T01:00:11.145 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2024-09-22T01:00:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQD3a+9mFXznOhAAF3Qq1qWduxaK+OO8TSH5bA== 2024-09-22T01:00:11.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T01:00:11.187 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T01:00:11.233 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.235 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.276 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.403 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.443 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.530 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T01:00:11.571 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T01:00:11.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.704 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.739 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.784 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.867 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.911 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:00:11.956 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T01:00:11.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T01:00:12.049 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T01:00:12.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T01:00:12.126 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-22T01:00:12.126 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-22T01:00:12.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T01:00:12.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-22T01:00:12.285 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-22T01:00:12.335 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T01:00:12.426 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-22T01:00:12.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-22T01:00:12.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T01:00:12.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-22T01:00:12.558 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:00:12.561 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:00:12.604 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-22T01:00:12.604 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-22T01:00:12.604 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-22T01:00:12.607 INFO:tasks.ceph.mon.a:Started 2024-09-22T01:00:12.607 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-22T01:00:12.607 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-22T01:00:12.609 INFO:tasks.ceph.mon.c:Started 2024-09-22T01:00:12.610 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-22T01:00:12.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-22T01:00:12.650 INFO:tasks.ceph.mon.b:Started 2024-09-22T01:00:12.650 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-22T01:00:12.650 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-22T01:00:12.650 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-22T01:00:12.653 INFO:tasks.ceph.mgr.y:Started 2024-09-22T01:00:12.653 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-22T01:00:12.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-22T01:00:12.655 INFO:tasks.ceph.mgr.x:Started 2024-09-22T01:00:12.655 INFO:tasks.ceph:Setting crush tunables to default 2024-09-22T01:00:12.655 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd crush tunables default 2024-09-22T01:00:18.149 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2024-09-22T01:00:18.169 INFO:tasks.ceph:check_enable_crimson: False 2024-09-22T01:00:18.169 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-22T01:00:18.169 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:18.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-22T01:00:18.182 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:18.182 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-22T01:00:18.237 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:18.237 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-22T01:00:18.245 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:18.242+0000 7f517aa14180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-22T01:00:18.294 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-22T01:00:18.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-22T01:00:18.350 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:18.350 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-22T01:00:18.364 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:18.364 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-22T01:00:18.419 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:18.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-22T01:00:18.475 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-22T01:00:18.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-22T01:00:18.528 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 8c0f94a8-3c63-442d-9dc1-50c1278929da 0 2024-09-22T01:00:18.735 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:18.730+0000 7f517aa14180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-22T01:00:18.788 INFO:teuthology.orchestra.run.smithi119.stdout:0 2024-09-22T01:00:18.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 2d2cf30e-e9bd-40ba-8293-193faa02ea69 1 2024-09-22T01:00:18.956 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:18.950+0000 7f517aa14180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-22T01:00:18.997 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-09-22T01:00:19.009 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:19.005+0000 7f677362b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-22T01:00:19.014 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new a3d3e330-9ea1-4a5c-811b-f285f9a3cff4 2 2024-09-22T01:00:19.207 INFO:teuthology.orchestra.run.smithi119.stdout:2 2024-09-22T01:00:19.224 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 239072f0-4b69-4624-ba6c-733f299208da 3 2024-09-22T01:00:19.431 INFO:teuthology.orchestra.run.smithi119.stdout:3 2024-09-22T01:00:19.448 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new fd35cb24-eb2c-4c17-abe9-ce5c34db9014 4 2024-09-22T01:00:19.539 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:19.534+0000 7f517aa14180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-22T01:00:19.565 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:19.565+0000 7f677362b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-22T01:00:19.595 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:19.590+0000 7f517aa14180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-22T01:00:19.647 INFO:teuthology.orchestra.run.smithi119.stdout:4 2024-09-22T01:00:19.648 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:19.646+0000 7f517aa14180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-22T01:00:19.664 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new b60a584d-5334-41e4-92d9-f6689de6b8bf 5 2024-09-22T01:00:19.729 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:19.726+0000 7f517aa14180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-22T01:00:19.775 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:19.773+0000 7f677362b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-22T01:00:19.782 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:19.778+0000 7f517aa14180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-22T01:00:19.861 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:19.858+0000 7f517aa14180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-22T01:00:19.869 INFO:teuthology.orchestra.run.smithi119.stdout:5 2024-09-22T01:00:19.888 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 9be24894-1100-4e4e-bf1b-79e163fd97e0 6 2024-09-22T01:00:19.917 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:19.914+0000 7f517aa14180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-22T01:00:20.074 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:20.070+0000 7f517aa14180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-22T01:00:20.084 INFO:teuthology.orchestra.run.smithi119.stdout:6 2024-09-22T01:00:20.100 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 293cfbc3-8c2e-40b5-ab89-0a124cc8651c 7 2024-09-22T01:00:20.138 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:20.134+0000 7f517aa14180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-22T01:00:20.337 INFO:teuthology.orchestra.run.smithi119.stdout:7 2024-09-22T01:00:20.353 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T01:00:20.353 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-22T01:00:20.356 INFO:tasks.ceph.osd.0:Started 2024-09-22T01:00:20.356 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T01:00:20.356 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-22T01:00:20.358 INFO:tasks.ceph.osd.1:Started 2024-09-22T01:00:20.358 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T01:00:20.358 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-22T01:00:20.361 INFO:tasks.ceph.osd.2:Started 2024-09-22T01:00:20.361 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T01:00:20.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-22T01:00:20.362 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:20.361+0000 7f677362b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-22T01:00:20.363 INFO:tasks.ceph.osd.3:Started 2024-09-22T01:00:20.363 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-22T01:00:20.363 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-22T01:00:20.365 INFO:tasks.ceph.osd.4:Started 2024-09-22T01:00:20.365 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-22T01:00:20.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-22T01:00:20.368 INFO:tasks.ceph.osd.5:Started 2024-09-22T01:00:20.368 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-22T01:00:20.368 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-22T01:00:20.371 INFO:tasks.ceph.osd.6:Started 2024-09-22T01:00:20.371 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-22T01:00:20.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-22T01:00:20.373 INFO:tasks.ceph.osd.7:Started 2024-09-22T01:00:20.373 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:00:20.454 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:20.453+0000 7f677362b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-22T01:00:20.520 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:20.517+0000 7f677362b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-22T01:00:20.587 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:20.587 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:00:20.335765+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T01:00:20.335765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:00:20.603 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-22T01:00:20.603 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-22T01:00:20.603 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:20.601+0000 7f677362b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-22T01:00:20.657 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:20.657+0000 7f677362b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-22T01:00:20.738 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:20.737+0000 7f677362b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-22T01:00:20.767 INFO:tasks.ceph.mgr.x.smithi119.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-22T01:00:20.768 INFO:tasks.ceph.mgr.x.smithi119.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-22T01:00:20.768 INFO:tasks.ceph.mgr.x.smithi119.stderr: from numpy import show_config as show_numpy_config 2024-09-22T01:00:20.793 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:20.789+0000 7f677362b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-22T01:00:20.807 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:20.802+0000 7f517aa14180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-22T01:00:20.939 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:20.934+0000 7f517aa14180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-22T01:00:20.951 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:20.949+0000 7f677362b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-22T01:00:21.016 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:21.013+0000 7f677362b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-22T01:00:21.094 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:21.090+0000 7f517aa14180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-22T01:00:21.158 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:21.154+0000 7f517aa14180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-22T01:00:21.292 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:21.290+0000 7f517aa14180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-22T01:00:21.408 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T01:00:21.589 INFO:teuthology.misc.health.smithi002.stdout: 2024-09-22T01:00:21.589 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":10,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:00:20.335765+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T01:00:20.335765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:00:21.606 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-22T01:00:21.641 INFO:tasks.ceph.mgr.y.smithi002.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-22T01:00:21.641 INFO:tasks.ceph.mgr.y.smithi002.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-22T01:00:21.641 INFO:tasks.ceph.mgr.y.smithi002.stderr: from numpy import show_config as show_numpy_config 2024-09-22T01:00:21.685 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:21.685+0000 7f677362b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-22T01:00:21.832 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:21.829+0000 7f677362b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-22T01:00:21.899 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:21.894+0000 7f517aa14180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-22T01:00:21.945 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:21.942+0000 7f093064f7c0 -1 Falling back to public interface 2024-09-22T01:00:21.975 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:21.970+0000 7f517aa14180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-22T01:00:22.023 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:22.021+0000 7fc202f167c0 -1 Falling back to public interface 2024-09-22T01:00:22.024 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:22.021+0000 7f677362b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-22T01:00:22.038 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:22.037+0000 7f180b20b7c0 -1 Falling back to public interface 2024-09-22T01:00:22.062 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:22.061+0000 7fab42fce7c0 -1 Falling back to public interface 2024-09-22T01:00:22.093 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:22.089+0000 7f677362b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-22T01:00:22.157 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:22.154+0000 7f517aa14180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-22T01:00:22.217 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:22.214+0000 7f29036147c0 -1 Falling back to public interface 2024-09-22T01:00:22.231 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:22.226+0000 7f517aa14180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-22T01:00:22.234 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:22.233+0000 7f677362b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-22T01:00:22.271 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:22.269+0000 7f97d44a97c0 -1 Falling back to public interface 2024-09-22T01:00:22.481 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:22.478+0000 7f1f823807c0 -1 Falling back to public interface 2024-09-22T01:00:22.521 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:22.518+0000 7f517aa14180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-22T01:00:22.545 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:22.542+0000 7f62a14c67c0 -1 Falling back to public interface 2024-09-22T01:00:22.584 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:22.582+0000 7f517aa14180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-22T01:00:22.875 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:22.873+0000 7f677362b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-22T01:00:22.953 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:22.953+0000 7f677362b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-22T01:00:23.147 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:23.145+0000 7f677362b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-22T01:00:23.224 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:23.221+0000 7f677362b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-22T01:00:23.529 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:23.529+0000 7f677362b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-22T01:00:23.595 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-22T01:00:23.593+0000 7f677362b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-22T01:00:23.624 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:23.622+0000 7f5121548640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-22T01:00:23.626 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-22T01:00:23.622+0000 7f5121548640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-22T01:00:26.181 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:26.181+0000 7fab42fce7c0 -1 osd.1 0 log_to_monitors true 2024-09-22T01:00:26.269 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:26.266+0000 7f093064f7c0 -1 osd.4 0 log_to_monitors true 2024-09-22T01:00:26.534 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:26.530+0000 7f29036147c0 -1 osd.7 0 log_to_monitors true 2024-09-22T01:00:26.552 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:26.549+0000 7f97d44a97c0 -1 osd.0 0 log_to_monitors true 2024-09-22T01:00:26.752 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:26.749+0000 7fc202f167c0 -1 osd.2 0 log_to_monitors true 2024-09-22T01:00:27.141 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:27.141+0000 7f180b20b7c0 -1 osd.3 0 log_to_monitors true 2024-09-22T01:00:27.155 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:27.150+0000 7f1f823807c0 -1 osd.6 0 log_to_monitors true 2024-09-22T01:00:27.257 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:27.254+0000 7f62a14c67c0 -1 osd.5 0 log_to_monitors true 2024-09-22T01:00:28.412 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T01:00:28.849 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:28.849+0000 7fab3e754640 -1 osd.1 0 waiting for initial osdmap 2024-09-22T01:00:28.859 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:28.857+0000 7f97cfc2f640 -1 osd.0 0 waiting for initial osdmap 2024-09-22T01:00:28.886 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:28.885+0000 7fab36d9b640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:00:28.899 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:28.897+0000 7f97c8276640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:00:28.902 INFO:teuthology.misc.health.smithi002.stdout: 2024-09-22T01:00:28.902 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":13,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:00:28.643874+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T01:00:20.335765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:00:28.924 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-22T01:00:29.299 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:29.294+0000 7f092bdd5640 -1 osd.4 0 waiting for initial osdmap 2024-09-22T01:00:29.357 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:29.354+0000 7f092441c640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:00:29.372 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:29.370+0000 7f28ffdcf640 -1 osd.7 0 waiting for initial osdmap 2024-09-22T01:00:29.425 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:29.422+0000 7f28f73e1640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:00:29.787 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:29.785+0000 7fc1fe69c640 -1 osd.2 0 waiting for initial osdmap 2024-09-22T01:00:29.810 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:29.809+0000 7fc1f6ce3640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:00:29.817 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:29.817+0000 7f1806991640 -1 osd.3 0 waiting for initial osdmap 2024-09-22T01:00:29.842 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:29.841+0000 7f17fefd8640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:00:30.177 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:30.174+0000 7f1f7eb3b640 -1 osd.6 0 waiting for initial osdmap 2024-09-22T01:00:30.200 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:30.198+0000 7f1f7614d640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:00:30.257 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:30.254+0000 7f629cc4c640 -1 osd.5 0 waiting for initial osdmap 2024-09-22T01:00:30.293 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:30.290+0000 7f6295293640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:00:35.731 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T01:00:36.039 INFO:teuthology.misc.health.smithi002.stdout: 2024-09-22T01:00:36.039 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":17,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:00:32.881554+0000","last_up_change":"2024-09-22T01:00:30.876018+0000","last_in_change":"2024-09-22T01:00:20.335765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:00:30.699559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6825","nonce":2277223948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6827","nonce":2277223948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6831","nonce":2277223948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6829","nonce":2277223948}]},"public_addr":"172.21.15.2:6825/2277223948","cluster_addr":"172.21.15.2:6827/2277223948","heartbeat_back_addr":"172.21.15.2:6831/2277223948","heartbeat_front_addr":"172.21.15.2:6829/2277223948","state":["exists","up"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6817","nonce":463340194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6819","nonce":463340194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6823","nonce":463340194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6821","nonce":463340194}]},"public_addr":"172.21.15.2:6817/463340194","cluster_addr":"172.21.15.2:6819/463340194","heartbeat_back_addr":"172.21.15.2:6823/463340194","heartbeat_front_addr":"172.21.15.2:6821/463340194","state":["exists","up"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6801","nonce":387643397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6803","nonce":387643397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6807","nonce":387643397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6805","nonce":387643397}]},"public_addr":"172.21.15.2:6801/387643397","cluster_addr":"172.21.15.2:6803/387643397","heartbeat_back_addr":"172.21.15.2:6807/387643397","heartbeat_front_addr":"172.21.15.2:6805/387643397","state":["exists","up"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6809","nonce":1215210764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6811","nonce":1215210764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6815","nonce":1215210764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6813","nonce":1215210764}]},"public_addr":"172.21.15.2:6809/1215210764","cluster_addr":"172.21.15.2:6811/1215210764","heartbeat_back_addr":"172.21.15.2:6815/1215210764","heartbeat_front_addr":"172.21.15.2:6813/1215210764","state":["exists","up"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6801","nonce":2463696152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6803","nonce":2463696152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6807","nonce":2463696152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6805","nonce":2463696152}]},"public_addr":"172.21.15.119:6801/2463696152","cluster_addr":"172.21.15.119:6803/2463696152","heartbeat_back_addr":"172.21.15.119:6807/2463696152","heartbeat_front_addr":"172.21.15.119:6805/2463696152","state":["exists","up"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6825","nonce":1388509479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6827","nonce":1388509479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6831","nonce":1388509479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6829","nonce":1388509479}]},"public_addr":"172.21.15.119:6825/1388509479","cluster_addr":"172.21.15.119:6827/1388509479","heartbeat_back_addr":"172.21.15.119:6831/1388509479","heartbeat_front_addr":"172.21.15.119:6829/1388509479","state":["exists","up"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6817","nonce":4220217922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6819","nonce":4220217922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6823","nonce":4220217922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6821","nonce":4220217922}]},"public_addr":"172.21.15.119:6817/4220217922","cluster_addr":"172.21.15.119:6819/4220217922","heartbeat_back_addr":"172.21.15.119:6823/4220217922","heartbeat_front_addr":"172.21.15.119:6821/4220217922","state":["exists","up"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6809","nonce":3867156106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6811","nonce":3867156106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6815","nonce":3867156106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6813","nonce":3867156106}]},"public_addr":"172.21.15.119:6809/3867156106","cluster_addr":"172.21.15.119:6811/3867156106","heartbeat_back_addr":"172.21.15.119:6815/3867156106","heartbeat_front_addr":"172.21.15.119:6813/3867156106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.556128+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.159753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.766199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.155529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.226232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.243346+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.143084+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.524369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:00:36.056 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-22T01:00:36.056 INFO:tasks.ceph:Creating RBD pool 2024-09-22T01:00:36.057 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-22T01:00:36.910 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'rbd' created 2024-09-22T01:00:36.940 DEBUG:teuthology.orchestra.run.smithi002:> rbd --cluster ceph pool init rbd 2024-09-22T01:00:39.965 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-22T01:00:39.965 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-22T01:00:39.965 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-22T01:00:39.965 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-22T01:00:39.966 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-22T01:00:40.323 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:40.341 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":6,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1369806497},{"type":"v1","addr":"172.21.15.119:6833","nonce":1369806497}]},"active_addr":"172.21.15.119:6833/1369806497","active_change":"2024-09-22T01:00:22.607096+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1269100006}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2232135440}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":927940871}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3044835228}]}]} 2024-09-22T01:00:40.344 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-22T01:00:40.344 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T01:00:40.344 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:00:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:40.644 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":21,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:00:39.930825+0000","last_up_change":"2024-09-22T01:00:30.876018+0000","last_in_change":"2024-09-22T01:00:20.335765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:00:30.699559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:00:36.410605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6825","nonce":2277223948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6827","nonce":2277223948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6831","nonce":2277223948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6829","nonce":2277223948}]},"public_addr":"172.21.15.2:6825/2277223948","cluster_addr":"172.21.15.2:6827/2277223948","heartbeat_back_addr":"172.21.15.2:6831/2277223948","heartbeat_front_addr":"172.21.15.2:6829/2277223948","state":["exists","up"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6817","nonce":463340194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6819","nonce":463340194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6823","nonce":463340194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6821","nonce":463340194}]},"public_addr":"172.21.15.2:6817/463340194","cluster_addr":"172.21.15.2:6819/463340194","heartbeat_back_addr":"172.21.15.2:6823/463340194","heartbeat_front_addr":"172.21.15.2:6821/463340194","state":["exists","up"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6801","nonce":387643397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6803","nonce":387643397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6807","nonce":387643397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6805","nonce":387643397}]},"public_addr":"172.21.15.2:6801/387643397","cluster_addr":"172.21.15.2:6803/387643397","heartbeat_back_addr":"172.21.15.2:6807/387643397","heartbeat_front_addr":"172.21.15.2:6805/387643397","state":["exists","up"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6809","nonce":1215210764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6811","nonce":1215210764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6815","nonce":1215210764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6813","nonce":1215210764}]},"public_addr":"172.21.15.2:6809/1215210764","cluster_addr":"172.21.15.2:6811/1215210764","heartbeat_back_addr":"172.21.15.2:6815/1215210764","heartbeat_front_addr":"172.21.15.2:6813/1215210764","state":["exists","up"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6801","nonce":2463696152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6803","nonce":2463696152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6807","nonce":2463696152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6805","nonce":2463696152}]},"public_addr":"172.21.15.119:6801/2463696152","cluster_addr":"172.21.15.119:6803/2463696152","heartbeat_back_addr":"172.21.15.119:6807/2463696152","heartbeat_front_addr":"172.21.15.119:6805/2463696152","state":["exists","up"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6825","nonce":1388509479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6827","nonce":1388509479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6831","nonce":1388509479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6829","nonce":1388509479}]},"public_addr":"172.21.15.119:6825/1388509479","cluster_addr":"172.21.15.119:6827/1388509479","heartbeat_back_addr":"172.21.15.119:6831/1388509479","heartbeat_front_addr":"172.21.15.119:6829/1388509479","state":["exists","up"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6817","nonce":4220217922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6819","nonce":4220217922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6823","nonce":4220217922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6821","nonce":4220217922}]},"public_addr":"172.21.15.119:6817/4220217922","cluster_addr":"172.21.15.119:6819/4220217922","heartbeat_back_addr":"172.21.15.119:6823/4220217922","heartbeat_front_addr":"172.21.15.119:6821/4220217922","state":["exists","up"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6809","nonce":3867156106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6811","nonce":3867156106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6815","nonce":3867156106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6813","nonce":3867156106}]},"public_addr":"172.21.15.119:6809/3867156106","cluster_addr":"172.21.15.119:6811/3867156106","heartbeat_back_addr":"172.21.15.119:6815/3867156106","heartbeat_front_addr":"172.21.15.119:6813/3867156106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.556128+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.159753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.766199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.155529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.226232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.243346+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.143084+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.524369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:00:40.665 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T01:00:40.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:00:40.973 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:40.973 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":21,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:00:39.930825+0000","last_up_change":"2024-09-22T01:00:30.876018+0000","last_in_change":"2024-09-22T01:00:20.335765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:00:30.699559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:00:36.410605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6825","nonce":2277223948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6827","nonce":2277223948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6831","nonce":2277223948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6829","nonce":2277223948}]},"public_addr":"172.21.15.2:6825/2277223948","cluster_addr":"172.21.15.2:6827/2277223948","heartbeat_back_addr":"172.21.15.2:6831/2277223948","heartbeat_front_addr":"172.21.15.2:6829/2277223948","state":["exists","up"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6817","nonce":463340194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6819","nonce":463340194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6823","nonce":463340194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6821","nonce":463340194}]},"public_addr":"172.21.15.2:6817/463340194","cluster_addr":"172.21.15.2:6819/463340194","heartbeat_back_addr":"172.21.15.2:6823/463340194","heartbeat_front_addr":"172.21.15.2:6821/463340194","state":["exists","up"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6801","nonce":387643397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6803","nonce":387643397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6807","nonce":387643397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6805","nonce":387643397}]},"public_addr":"172.21.15.2:6801/387643397","cluster_addr":"172.21.15.2:6803/387643397","heartbeat_back_addr":"172.21.15.2:6807/387643397","heartbeat_front_addr":"172.21.15.2:6805/387643397","state":["exists","up"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6809","nonce":1215210764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6811","nonce":1215210764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6815","nonce":1215210764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6813","nonce":1215210764}]},"public_addr":"172.21.15.2:6809/1215210764","cluster_addr":"172.21.15.2:6811/1215210764","heartbeat_back_addr":"172.21.15.2:6815/1215210764","heartbeat_front_addr":"172.21.15.2:6813/1215210764","state":["exists","up"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6801","nonce":2463696152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6803","nonce":2463696152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6807","nonce":2463696152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6805","nonce":2463696152}]},"public_addr":"172.21.15.119:6801/2463696152","cluster_addr":"172.21.15.119:6803/2463696152","heartbeat_back_addr":"172.21.15.119:6807/2463696152","heartbeat_front_addr":"172.21.15.119:6805/2463696152","state":["exists","up"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6825","nonce":1388509479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6827","nonce":1388509479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6831","nonce":1388509479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1388509479},{"type":"v1","addr":"172.21.15.119:6829","nonce":1388509479}]},"public_addr":"172.21.15.119:6825/1388509479","cluster_addr":"172.21.15.119:6827/1388509479","heartbeat_back_addr":"172.21.15.119:6831/1388509479","heartbeat_front_addr":"172.21.15.119:6829/1388509479","state":["exists","up"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6817","nonce":4220217922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6819","nonce":4220217922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6823","nonce":4220217922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6821","nonce":4220217922}]},"public_addr":"172.21.15.119:6817/4220217922","cluster_addr":"172.21.15.119:6819/4220217922","heartbeat_back_addr":"172.21.15.119:6823/4220217922","heartbeat_front_addr":"172.21.15.119:6821/4220217922","state":["exists","up"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6809","nonce":3867156106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6811","nonce":3867156106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6815","nonce":3867156106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6813","nonce":3867156106}]},"public_addr":"172.21.15.119:6809/3867156106","cluster_addr":"172.21.15.119:6811/3867156106","heartbeat_back_addr":"172.21.15.119:6815/3867156106","heartbeat_front_addr":"172.21.15.119:6813/3867156106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.556128+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.159753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.766199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.155529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.226232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.243346+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.143084+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.524369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:00:40.992 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T01:00:40.993 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T01:00:40.993 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T01:00:40.993 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T01:00:40.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-22T01:00:40.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-22T01:00:40.995 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-22T01:00:40.995 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-22T01:00:41.240 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-22T01:00:41.240 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-22T01:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-22T01:00:41.248 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T01:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout:64424509445 2024-09-22T01:00:41.262 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-22T01:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout:60129542148 2024-09-22T01:00:41.263 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout:64424509444 2024-09-22T01:00:41.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-22T01:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-22T01:00:41.276 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-22T01:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-22T01:00:41.279 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T01:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout:60129542151 2024-09-22T01:00:41.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T01:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout:60129542148 2024-09-22T01:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout:60129542146 2024-09-22T01:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout:64424509443 2024-09-22T01:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout:60129542147 2024-09-22T01:00:41.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.5 2024-09-22T01:00:41.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542146 for osd.3 2024-09-22T01:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout:64424509444 2024-09-22T01:00:41.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.4 2024-09-22T01:00:41.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.2 2024-09-22T01:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout:60129542148 2024-09-22T01:00:41.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.6 2024-09-22T01:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-22T01:00:41.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.7 2024-09-22T01:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout:60129542148 2024-09-22T01:00:41.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.1 2024-09-22T01:00:41.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.0 2024-09-22T01:00:42.812 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-22T01:00:42.816 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:00:42.826 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-22T01:00:42.831 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T01:00:42.844 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-22T01:00:42.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-22T01:00:42.868 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T01:00:42.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T01:00:43.351 INFO:teuthology.orchestra.run.smithi002.stdout:64424509445 2024-09-22T01:00:43.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.5 2024-09-22T01:00:43.376 DEBUG:teuthology.parallel:result is None 2024-09-22T01:00:43.382 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-22T01:00:43.398 INFO:teuthology.orchestra.run.smithi002.stdout:60129542148 2024-09-22T01:00:43.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2024-09-22T01:00:43.400 DEBUG:teuthology.parallel:result is None 2024-09-22T01:00:43.405 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-22T01:00:43.412 INFO:teuthology.orchestra.run.smithi002.stdout:64424509446 2024-09-22T01:00:43.417 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-22T01:00:43.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.3 2024-09-22T01:00:43.426 DEBUG:teuthology.parallel:result is None 2024-09-22T01:00:43.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-09-22T01:00:43.430 DEBUG:teuthology.parallel:result is None 2024-09-22T01:00:43.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.6 2024-09-22T01:00:43.439 DEBUG:teuthology.parallel:result is None 2024-09-22T01:00:43.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-09-22T01:00:43.439 DEBUG:teuthology.parallel:result is None 2024-09-22T01:00:43.439 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-22T01:00:43.446 INFO:teuthology.orchestra.run.smithi002.stdout:60129542151 2024-09-22T01:00:43.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-09-22T01:00:43.465 DEBUG:teuthology.parallel:result is None 2024-09-22T01:00:43.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2024-09-22T01:00:43.471 DEBUG:teuthology.parallel:result is None 2024-09-22T01:00:43.471 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T01:00:43.471 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:00:43.801 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:43.801 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:00:43.817 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-22T01:00:42.619143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":248184,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726536,"statfs":{"total":773094113280,"available":772839972864,"internally_reserved":0,"allocated":2867200,"data_stored":1752110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.706489"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.193083+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:00:40.193083+0000","last_peered":"2024-09-22T01:00:40.193083+0000","last_clean":"2024-09-22T01:00:40.193083+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:00:40.193083+0000","last_undegraded":"2024-09-22T01:00:40.193083+0000","last_fullsized":"2024-09-22T01:00:40.193083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946521+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:00:39.946521+0000","last_peered":"2024-09-22T01:00:39.946521+0000","last_clean":"2024-09-22T01:00:39.946521+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:00:39.946521+0000","last_undegraded":"2024-09-22T01:00:39.946521+0000","last_fullsized":"2024-09-22T01:00:39.946521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354336+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:00:40.354336+0000","last_peered":"2024-09-22T01:00:40.354336+0000","last_clean":"2024-09-22T01:00:40.354336+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:00:40.354336+0000","last_undegraded":"2024-09-22T01:00:40.354336+0000","last_fullsized":"2024-09-22T01:00:40.354336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946358+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:00:39.946358+0000","last_peered":"2024-09-22T01:00:39.946358+0000","last_clean":"2024-09-22T01:00:39.946358+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:00:39.946358+0000","last_undegraded":"2024-09-22T01:00:39.946358+0000","last_fullsized":"2024-09-22T01:00:39.946358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.951945+0000","last_change":"2024-09-22T01:00:39.947537+0000","last_active":"2024-09-22T01:00:39.951945+0000","last_peered":"2024-09-22T01:00:39.951945+0000","last_clean":"2024-09-22T01:00:39.951945+0000","last_became_active":"2024-09-22T01:00:37.928490+0000","last_became_peered":"2024-09-22T01:00:37.928490+0000","last_unstale":"2024-09-22T01:00:39.951945+0000","last_undegraded":"2024-09-22T01:00:39.951945+0000","last_fullsized":"2024-09-22T01:00:39.951945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:36.384733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.945210+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:00:39.945210+0000","last_peered":"2024-09-22T01:00:39.945210+0000","last_clean":"2024-09-22T01:00:39.945210+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:00:39.945210+0000","last_undegraded":"2024-09-22T01:00:39.945210+0000","last_fullsized":"2024-09-22T01:00:39.945210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354460+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:00:40.354460+0000","last_peered":"2024-09-22T01:00:40.354460+0000","last_clean":"2024-09-22T01:00:40.354460+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:00:40.354460+0000","last_undegraded":"2024-09-22T01:00:40.354460+0000","last_fullsized":"2024-09-22T01:00:40.354460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.947793+0000","last_change":"2024-09-22T01:00:39.947793+0000","last_active":"2024-09-22T01:00:39.947793+0000","last_peered":"2024-09-22T01:00:39.947793+0000","last_clean":"2024-09-22T01:00:39.947793+0000","last_became_active":"2024-09-22T01:00:37.927774+0000","last_became_peered":"2024-09-22T01:00:37.927774+0000","last_unstale":"2024-09-22T01:00:39.947793+0000","last_undegraded":"2024-09-22T01:00:39.947793+0000","last_fullsized":"2024-09-22T01:00:39.947793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:33.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088854699999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.353465+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:00:40.353465+0000","last_peered":"2024-09-22T01:00:40.353465+0000","last_clean":"2024-09-22T01:00:40.353465+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:00:40.353465+0000","last_undegraded":"2024-09-22T01:00:40.353465+0000","last_fullsized":"2024-09-22T01:00:40.353465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:00:43.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:00:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:44.109 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:00:44.125 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-22T01:00:42.619143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":248184,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726536,"statfs":{"total":773094113280,"available":772839972864,"internally_reserved":0,"allocated":2867200,"data_stored":1752110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.706489"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.193083+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:00:40.193083+0000","last_peered":"2024-09-22T01:00:40.193083+0000","last_clean":"2024-09-22T01:00:40.193083+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:00:40.193083+0000","last_undegraded":"2024-09-22T01:00:40.193083+0000","last_fullsized":"2024-09-22T01:00:40.193083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946521+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:00:39.946521+0000","last_peered":"2024-09-22T01:00:39.946521+0000","last_clean":"2024-09-22T01:00:39.946521+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:00:39.946521+0000","last_undegraded":"2024-09-22T01:00:39.946521+0000","last_fullsized":"2024-09-22T01:00:39.946521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354336+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:00:40.354336+0000","last_peered":"2024-09-22T01:00:40.354336+0000","last_clean":"2024-09-22T01:00:40.354336+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:00:40.354336+0000","last_undegraded":"2024-09-22T01:00:40.354336+0000","last_fullsized":"2024-09-22T01:00:40.354336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946358+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:00:39.946358+0000","last_peered":"2024-09-22T01:00:39.946358+0000","last_clean":"2024-09-22T01:00:39.946358+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:00:39.946358+0000","last_undegraded":"2024-09-22T01:00:39.946358+0000","last_fullsized":"2024-09-22T01:00:39.946358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.951945+0000","last_change":"2024-09-22T01:00:39.947537+0000","last_active":"2024-09-22T01:00:39.951945+0000","last_peered":"2024-09-22T01:00:39.951945+0000","last_clean":"2024-09-22T01:00:39.951945+0000","last_became_active":"2024-09-22T01:00:37.928490+0000","last_became_peered":"2024-09-22T01:00:37.928490+0000","last_unstale":"2024-09-22T01:00:39.951945+0000","last_undegraded":"2024-09-22T01:00:39.951945+0000","last_fullsized":"2024-09-22T01:00:39.951945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:36.384733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.945210+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:00:39.945210+0000","last_peered":"2024-09-22T01:00:39.945210+0000","last_clean":"2024-09-22T01:00:39.945210+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:00:39.945210+0000","last_undegraded":"2024-09-22T01:00:39.945210+0000","last_fullsized":"2024-09-22T01:00:39.945210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354460+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:00:40.354460+0000","last_peered":"2024-09-22T01:00:40.354460+0000","last_clean":"2024-09-22T01:00:40.354460+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:00:40.354460+0000","last_undegraded":"2024-09-22T01:00:40.354460+0000","last_fullsized":"2024-09-22T01:00:40.354460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.947793+0000","last_change":"2024-09-22T01:00:39.947793+0000","last_active":"2024-09-22T01:00:39.947793+0000","last_peered":"2024-09-22T01:00:39.947793+0000","last_clean":"2024-09-22T01:00:39.947793+0000","last_became_active":"2024-09-22T01:00:37.927774+0000","last_became_peered":"2024-09-22T01:00:37.927774+0000","last_unstale":"2024-09-22T01:00:39.947793+0000","last_undegraded":"2024-09-22T01:00:39.947793+0000","last_fullsized":"2024-09-22T01:00:39.947793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:33.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088854699999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.353465+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:00:40.353465+0000","last_peered":"2024-09-22T01:00:40.353465+0000","last_clean":"2024-09-22T01:00:40.353465+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:00:40.353465+0000","last_undegraded":"2024-09-22T01:00:40.353465+0000","last_fullsized":"2024-09-22T01:00:40.353465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:00:44.126 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T01:00:44.126 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-22T01:00:44.126 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-22T01:00:44.126 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-22T01:00:44.457 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:44.457 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-22T01:00:44.475 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-22T01:00:44.475 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-22T01:00:44.491 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-22T01:00:44.491 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-22T01:00:44.492 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-22T01:00:44.492 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-22T01:00:44.811 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-22T01:00:44.812 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T01:00:44.812 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:00:45.102 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:45.102 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:00:45.119 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-22T01:00:44.619763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":248184,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726536,"statfs":{"total":773094113280,"available":772839972864,"internally_reserved":0,"allocated":2867200,"data_stored":1752110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.704169"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.193083+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:00:40.193083+0000","last_peered":"2024-09-22T01:00:40.193083+0000","last_clean":"2024-09-22T01:00:40.193083+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:00:40.193083+0000","last_undegraded":"2024-09-22T01:00:40.193083+0000","last_fullsized":"2024-09-22T01:00:40.193083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946521+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:00:39.946521+0000","last_peered":"2024-09-22T01:00:39.946521+0000","last_clean":"2024-09-22T01:00:39.946521+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:00:39.946521+0000","last_undegraded":"2024-09-22T01:00:39.946521+0000","last_fullsized":"2024-09-22T01:00:39.946521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354336+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:00:40.354336+0000","last_peered":"2024-09-22T01:00:40.354336+0000","last_clean":"2024-09-22T01:00:40.354336+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:00:40.354336+0000","last_undegraded":"2024-09-22T01:00:40.354336+0000","last_fullsized":"2024-09-22T01:00:40.354336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946358+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:00:39.946358+0000","last_peered":"2024-09-22T01:00:39.946358+0000","last_clean":"2024-09-22T01:00:39.946358+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:00:39.946358+0000","last_undegraded":"2024-09-22T01:00:39.946358+0000","last_fullsized":"2024-09-22T01:00:39.946358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.951945+0000","last_change":"2024-09-22T01:00:39.947537+0000","last_active":"2024-09-22T01:00:39.951945+0000","last_peered":"2024-09-22T01:00:39.951945+0000","last_clean":"2024-09-22T01:00:39.951945+0000","last_became_active":"2024-09-22T01:00:37.928490+0000","last_became_peered":"2024-09-22T01:00:37.928490+0000","last_unstale":"2024-09-22T01:00:39.951945+0000","last_undegraded":"2024-09-22T01:00:39.951945+0000","last_fullsized":"2024-09-22T01:00:39.951945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:36.384733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.945210+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:00:39.945210+0000","last_peered":"2024-09-22T01:00:39.945210+0000","last_clean":"2024-09-22T01:00:39.945210+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:00:39.945210+0000","last_undegraded":"2024-09-22T01:00:39.945210+0000","last_fullsized":"2024-09-22T01:00:39.945210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354460+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:00:40.354460+0000","last_peered":"2024-09-22T01:00:40.354460+0000","last_clean":"2024-09-22T01:00:40.354460+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:00:40.354460+0000","last_undegraded":"2024-09-22T01:00:40.354460+0000","last_fullsized":"2024-09-22T01:00:40.354460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.947793+0000","last_change":"2024-09-22T01:00:39.947793+0000","last_active":"2024-09-22T01:00:39.947793+0000","last_peered":"2024-09-22T01:00:39.947793+0000","last_clean":"2024-09-22T01:00:39.947793+0000","last_became_active":"2024-09-22T01:00:37.927774+0000","last_became_peered":"2024-09-22T01:00:37.927774+0000","last_unstale":"2024-09-22T01:00:39.947793+0000","last_undegraded":"2024-09-22T01:00:39.947793+0000","last_fullsized":"2024-09-22T01:00:39.947793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:33.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088854699999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.353465+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:00:40.353465+0000","last_peered":"2024-09-22T01:00:40.353465+0000","last_clean":"2024-09-22T01:00:40.353465+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:00:40.353465+0000","last_undegraded":"2024-09-22T01:00:40.353465+0000","last_fullsized":"2024-09-22T01:00:40.353465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:00:45.120 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:00:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:45.417 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:00:45.436 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-22T01:00:44.619763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":248184,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726536,"statfs":{"total":773094113280,"available":772839972864,"internally_reserved":0,"allocated":2867200,"data_stored":1752110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.704169"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.193083+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:00:40.193083+0000","last_peered":"2024-09-22T01:00:40.193083+0000","last_clean":"2024-09-22T01:00:40.193083+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:00:40.193083+0000","last_undegraded":"2024-09-22T01:00:40.193083+0000","last_fullsized":"2024-09-22T01:00:40.193083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946521+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:00:39.946521+0000","last_peered":"2024-09-22T01:00:39.946521+0000","last_clean":"2024-09-22T01:00:39.946521+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:00:39.946521+0000","last_undegraded":"2024-09-22T01:00:39.946521+0000","last_fullsized":"2024-09-22T01:00:39.946521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354336+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:00:40.354336+0000","last_peered":"2024-09-22T01:00:40.354336+0000","last_clean":"2024-09-22T01:00:40.354336+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:00:40.354336+0000","last_undegraded":"2024-09-22T01:00:40.354336+0000","last_fullsized":"2024-09-22T01:00:40.354336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946358+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:00:39.946358+0000","last_peered":"2024-09-22T01:00:39.946358+0000","last_clean":"2024-09-22T01:00:39.946358+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:00:39.946358+0000","last_undegraded":"2024-09-22T01:00:39.946358+0000","last_fullsized":"2024-09-22T01:00:39.946358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.951945+0000","last_change":"2024-09-22T01:00:39.947537+0000","last_active":"2024-09-22T01:00:39.951945+0000","last_peered":"2024-09-22T01:00:39.951945+0000","last_clean":"2024-09-22T01:00:39.951945+0000","last_became_active":"2024-09-22T01:00:37.928490+0000","last_became_peered":"2024-09-22T01:00:37.928490+0000","last_unstale":"2024-09-22T01:00:39.951945+0000","last_undegraded":"2024-09-22T01:00:39.951945+0000","last_fullsized":"2024-09-22T01:00:39.951945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:36.384733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.945210+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:00:39.945210+0000","last_peered":"2024-09-22T01:00:39.945210+0000","last_clean":"2024-09-22T01:00:39.945210+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:00:39.945210+0000","last_undegraded":"2024-09-22T01:00:39.945210+0000","last_fullsized":"2024-09-22T01:00:39.945210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354460+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:00:40.354460+0000","last_peered":"2024-09-22T01:00:40.354460+0000","last_clean":"2024-09-22T01:00:40.354460+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:00:40.354460+0000","last_undegraded":"2024-09-22T01:00:40.354460+0000","last_fullsized":"2024-09-22T01:00:40.354460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.947793+0000","last_change":"2024-09-22T01:00:39.947793+0000","last_active":"2024-09-22T01:00:39.947793+0000","last_peered":"2024-09-22T01:00:39.947793+0000","last_clean":"2024-09-22T01:00:39.947793+0000","last_became_active":"2024-09-22T01:00:37.927774+0000","last_became_peered":"2024-09-22T01:00:37.927774+0000","last_unstale":"2024-09-22T01:00:39.947793+0000","last_undegraded":"2024-09-22T01:00:39.947793+0000","last_fullsized":"2024-09-22T01:00:39.947793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:33.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088854699999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.353465+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:00:40.353465+0000","last_peered":"2024-09-22T01:00:40.353465+0000","last_clean":"2024-09-22T01:00:40.353465+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:00:40.353465+0000","last_undegraded":"2024-09-22T01:00:40.353465+0000","last_fullsized":"2024-09-22T01:00:40.353465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:00:45.436 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T01:00:45.437 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-22T01:00:45.741 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 21 2024-09-22T01:00:45.741 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 7d08b428-4185-4f0f-a0fc-f7dc1fefdce9 2024-09-22T01:00:45.741 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-09-22T01:00:18.015746+0000 2024-09-22T01:00:45.741 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-09-22T01:00:39.930825+0000 2024-09-22T01:00:45.741 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-22T01:00:45.741 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 5 2024-09-22T01:00:45.741 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client jewel 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release squid 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: removed_snaps_queue [2~1] 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 8 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6824/2277223948,v1:172.21.15.2:6825/2277223948] [v2:172.21.15.2:6826/2277223948,v1:172.21.15.2:6827/2277223948] exists,up 8c0f94a8-3c63-442d-9dc1-50c1278929da 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6816/463340194,v1:172.21.15.2:6817/463340194] [v2:172.21.15.2:6818/463340194,v1:172.21.15.2:6819/463340194] exists,up 2d2cf30e-e9bd-40ba-8293-193faa02ea69 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6800/387643397,v1:172.21.15.2:6801/387643397] [v2:172.21.15.2:6802/387643397,v1:172.21.15.2:6803/387643397] exists,up a3d3e330-9ea1-4a5c-811b-f285f9a3cff4 2024-09-22T01:00:45.742 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6808/1215210764,v1:172.21.15.2:6809/1215210764] [v2:172.21.15.2:6810/1215210764,v1:172.21.15.2:6811/1215210764] exists,up 239072f0-4b69-4624-ba6c-733f299208da 2024-09-22T01:00:45.743 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/2463696152,v1:172.21.15.119:6801/2463696152] [v2:172.21.15.119:6802/2463696152,v1:172.21.15.119:6803/2463696152] exists,up fd35cb24-eb2c-4c17-abe9-ce5c34db9014 2024-09-22T01:00:45.743 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/1388509479,v1:172.21.15.119:6825/1388509479] [v2:172.21.15.119:6826/1388509479,v1:172.21.15.119:6827/1388509479] exists,up b60a584d-5334-41e4-92d9-f6689de6b8bf 2024-09-22T01:00:45.743 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/4220217922,v1:172.21.15.119:6817/4220217922] [v2:172.21.15.119:6818/4220217922,v1:172.21.15.119:6819/4220217922] exists,up 9be24894-1100-4e4e-bf1b-79e163fd97e0 2024-09-22T01:00:45.743 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/3867156106,v1:172.21.15.119:6809/3867156106] [v2:172.21.15.119:6810/3867156106,v1:172.21.15.119:6811/3867156106] exists,up 293cfbc3-8c2e-40b5-ab89-0a124cc8651c 2024-09-22T01:00:45.761 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6824/2277223948,v1:172.21.15.2:6825/2277223948] [v2:172.21.15.2:6826/2277223948,v1:172.21.15.2:6827/2277223948] exists,up 8c0f94a8-3c63-442d-9dc1-50c1278929da', 'osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6816/463340194,v1:172.21.15.2:6817/463340194] [v2:172.21.15.2:6818/463340194,v1:172.21.15.2:6819/463340194] exists,up 2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'osd.2 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6800/387643397,v1:172.21.15.2:6801/387643397] [v2:172.21.15.2:6802/387643397,v1:172.21.15.2:6803/387643397] exists,up a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6808/1215210764,v1:172.21.15.2:6809/1215210764] [v2:172.21.15.2:6810/1215210764,v1:172.21.15.2:6811/1215210764] exists,up 239072f0-4b69-4624-ba6c-733f299208da', 'osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/2463696152,v1:172.21.15.119:6801/2463696152] [v2:172.21.15.119:6802/2463696152,v1:172.21.15.119:6803/2463696152] exists,up fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/1388509479,v1:172.21.15.119:6825/1388509479] [v2:172.21.15.119:6826/1388509479,v1:172.21.15.119:6827/1388509479] exists,up b60a584d-5334-41e4-92d9-f6689de6b8bf', 'osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/4220217922,v1:172.21.15.119:6817/4220217922] [v2:172.21.15.119:6818/4220217922,v1:172.21.15.119:6819/4220217922] exists,up 9be24894-1100-4e4e-bf1b-79e163fd97e0', 'osd.7 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/3867156106,v1:172.21.15.119:6809/3867156106] [v2:172.21.15.119:6810/3867156106,v1:172.21.15.119:6811/3867156106] exists,up 293cfbc3-8c2e-40b5-ab89-0a124cc8651c'] 2024-09-22T01:00:45.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-22T01:00:45.883 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mon.a", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster": "ceph", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "admin_socket_mode": "", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_client_required": "cephx, none", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_cluster_required": "cephx", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_debug": "true", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_service_required": "cephx", 2024-09-22T01:00:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_supported": "cephx", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio": "true", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_poll_ms": "250", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_reap_max": "16", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_async_discard": "false", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_block_size": "4096", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio": "false", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_inflight_ios": "false", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_enable_discard": "false", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_flock_retry": "3", 2024-09-22T01:00:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_inject_crash": "0", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring": "false", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring_hipri": "false", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_type": "", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_alloc_size": "1048576", 2024-09-22T01:00:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_allocator": "hybrid", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_buffered_io": "true", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_for_zeros": "false", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_compact_log_sync": "false", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_max_log_runway": "4194304", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_max_prefetch": "1048576", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_min_flush_size": "524288", 2024-09-22T01:00:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_min_log_runway": "1048576", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_replay_recovery": "false", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_sync_write": "false", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_allocation_from_file": "true", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_allocator": "hybrid", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-22T01:00:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bdev_label_multi": "true", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_create": "true", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_create": "false", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_path": "", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_size": "0", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_path": "", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_preallocate_file": "false", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_size": "107374182400", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_create": "false", 2024-09-22T01:00:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_path": "", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_size": "100663296", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs": "true", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_autotune": "true", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-22T01:00:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size": "0", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_type": "2q", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_clone_cow": "true", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-22T01:00:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_mode": "none", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_csum_type": "crc32c", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_freelist": "false", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-22T01:00:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_small_allocations": "0", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_default_buffered_read": "true", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_default_buffered_write": "false", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-22T01:00:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fail_eio": "false", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-22T01:00:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mount": "false", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_umount": "false", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_ignore_data_csum": "false", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-22T01:00:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_op_age": "5.000000", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_alloc_size": "0", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size": "0", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_deferred_txc": "32", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size": "0", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_nid_prealloc": "1024", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-22T01:00:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_retry_disk_reads": "3", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_cf": "true", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-22T01:00:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_mem": "512", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_tracing": "false", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-22T01:00:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_zero_block_detection": "false", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-22T01:00:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_cluster_require_signatures": "false", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_cluster_require_version": "2", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_require_signatures": "false", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_require_version": "2", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_service_require_signatures": "false", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_service_require_version": "2", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_sign_messages": "true", 2024-09-22T01:00:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "chdir": "", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_acl_type": "", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_asio_thread_count": "2", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_cache_mid": "0.750000", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_cache_size": "16384", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_caps_release_delay": "5", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_check_pool_perm": "true", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_force_sync_read": "false", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_getattr_caps": "false", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_inject_features": "", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_inject_tick_delay": "0", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_die_on_failed_remount": "false", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_dirsize_rbytes": "true", 2024-09-22T01:00:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_force_lazyio": "false", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_fs": "", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_inject_release_failure": "false", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_max_inline_size": "4096", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mds_namespace": "", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_metadata": "", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_gid": "-1", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_timeout": "300", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_uid": "-1", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mountpoint": "/", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_notify_timeout": "10", 2024-09-22T01:00:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc": "true", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_dirty": "104857600", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_objects": "1000", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_size": "209715200", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_target_dirty": "8388608", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_permissions": "true", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_quota": "true", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_quota_df": "true", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_max_bytes": "0", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_max_periods": "4", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_min": "131072", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_reconnect_stale": "false", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_shutdown_timeout": "30", 2024-09-22T01:00:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_snapdir": ".snap", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_tick_interval": "1", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_trace": "", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_try_dentry_invalidate": "false", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_use_faked_inos": "false", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_use_random_mds": "false", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog": "false", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog_port": "12201", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_monitors": "default=true", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog": "false", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog_level": "info", 2024-09-22T01:00:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "-", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_network": "", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_network_interface": "", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_isal": "false", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_level": "5", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_winsize": "-15", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zstd_level": "1", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_alien_op_num_threads": "6", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_osd_stat_interval": "0", 2024-09-22T01:00:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_seastar_cpu_cores": "", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_seastar_num_threads": "0", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location": "", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location_hook": "", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location_hook_timeout": "10", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "daemonize": "false", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "daos_pool": "tank", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_alienstore": "0/5", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_allow_any_pool_priority": "false", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asok": "1/5", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asok_assert_abort": "false", 2024-09-22T01:00:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asserts_on_shutdown": "true", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_auth": "20/20", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bdev": "1/3", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bluefs": "1/5", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bluestore": "1/5", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_buffer": "0/1", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_ceph_exporter": "1/5", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cephfs_mirror": "0/5", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cephsqlite": "0/5", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_client": "0/5", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_compressor": "1/5", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_context": "0/1", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_crimson_interrupt": "0/5", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_crush": "1/1", 2024-09-22T01:00:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_crypto": "1/5", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cyanstore": "0/5", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_deliberately_leak_memory": "false", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_disable_randomized_ping": "false", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_dpdk": "1/5", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_eventtrace": "1/5", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_filer": "0/1", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_filestore": "1/3", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_finisher": "1/1", 2024-09-22T01:00:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_fuse": "1/5", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_heartbeat_testing_span": "0", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_heartbeatmap": "1/5", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_javaclient": "1/5", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_journal": "1/3", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_journaler": "0/5", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_kstore": "1/5", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_lockdep": "0/1", 2024-09-22T01:00:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mclock": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_balancer": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_locker": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_log": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_log_expire": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_migrator": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_quiesce": "3/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_memstore": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mgr": "2/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mgrc": "1/5", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mon": "20/20", 2024-09-22T01:00:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_monc": "0/10", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_ms": "1/1", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_none": "0/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objclass": "0/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objectcacher": "0/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objecter": "0/1", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_optracker": "0/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_osd": "1/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_paxos": "20/20", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_perfcounter": "1/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_prioritycache": "1/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rados": "0/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd": "0/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_mirror": "0/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_pwl": "0/5", 2024-09-22T01:00:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_replay": "0/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_refs": "0/0", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_reserver": "1/1", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw": "1/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_access": "1/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_datacache": "1/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_dbstore": "1/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_flight": "1/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_sync": "1/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rocksdb": "4/5", 2024-09-22T01:00:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_backref": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_cache": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_cleaner": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_device": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_epm": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_journal": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_lba": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_odata": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_omap": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_onode": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_t": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_tm": "0/5", 2024-09-22T01:00:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_striper": "0/1", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_test": "0/5", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_throttle": "1/1", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_timer": "0/1", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_tp": "0/5", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_trace": "1/5", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "device_failure_prediction_mode": "none", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_graylog": "false", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_journald": "false", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_stderr": "true", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_syslog": "false", 2024-09-22T01:00:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event_tracing": "false", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_addr": "0.0.0.0", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_http_port": "9926", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_prio_limit": "5", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_sort_metrics": "true", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_stats_period": "5", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "fake_statfs_for_testing": "0", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "fatal_signal_handlers": "true", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "filer_max_purge_ops": "10", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "filer_max_truncate_ops": "128", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_apply_finisher_threads": "1", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_blackhole": "false", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_btrfs_clone_range": "true", 2024-09-22T01:00:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_btrfs_snap": "true", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_caller_concurrency": "10", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_collect_device_partition_information": "true", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_commit_timeout": "600.000000", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_inject_read_err": "false", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_omap_check": "false", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_verify_split": "false", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_dump_file": "", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fadvise": "true", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fail_eio": "true", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fd_cache_shards": "16", 2024-09-22T01:00:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fd_cache_size": "128", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fiemap": "false", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fiemap_threshold": "4096", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_inject_stall": "0", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_parallel": "false", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_trailing": "false", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_writeahead": "false", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_kill_at": "0", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-22T01:00:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs": "0", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size": "0", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_merge_threshold": "-10", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_odsync_write": "false", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_backend": "rocksdb", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_backend_path": "", 2024-09-22T01:00:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_thread_timeout": "60", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_threads": "2", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_punch_hole": "false", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-22T01:00:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_ops": "50", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_seek_data_hole": "false", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_sloppy_crc": "false", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_splice": "false", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_split_multiple": "2", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_split_rand_factor": "20", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_update_to": "1000", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-22T01:00:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_enable": "true", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_xfs_extsize": "false", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_zfs_snap": "false", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "fio_dir": "/tmp/fio", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_allow_other": "true", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_atomic_o_trunc": "true", 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_big_writes": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_debug": "false", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_default_permissions": "false", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_disable_pagecache": "false", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_max_write": "0", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_multithreaded": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_require_active_mds": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_set_user_groups": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_move": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_read": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_write": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_use_invalidate_cb": "true", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-22T01:00:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "gss_target_name": "ceph", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_file": "", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_inject_failure": "0", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_interval": "5", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "host": "smithi002", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-22T01:00:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "inject_early_sigterm": "false", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "jaeger_agent_port": "6799", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "jaeger_tracing_enable": "false", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_aio": "true", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_align_min_size": "65536", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_block_align": "true", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_block_size": "4096", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_dio": "true", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_discard": "false", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_force_aio": "false", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_ignore_corruption": "false", 2024-09-22T01:00:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_max_write_bytes": "10485760", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_max_write_entries": "100", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_replay_from": "0", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_write_header_frequency": "0", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_zero_on_create": "false", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_prefetch_periods": "10", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_prezero_periods": "5", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_write_head_interval": "15", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "keyfile": "", 2024-09-22T01:00:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_backend": "rocksdb", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_default_stripe_size": "65536", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_fsck_on_mount": "false", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_max_bytes": "67108864", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_max_ops": "512", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_nid_prealloc": "1024", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_onode_map_size": "1024", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_sync_submit_transaction": "false", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_sync_transaction": "false", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "librados_thread_count": "2", 2024-09-22T01:00:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "lockdep": "false", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "lockdep_force_backtrace": "false", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_coarse_timestamps": "true", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_flush_on_exit": "false", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_graylog_host": "127.0.0.1", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_graylog_port": "12201", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_max_new": "1000", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_max_recent": "10000", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_stderr_prefix": "", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_stop_at_utilization": "0.970000", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": "true", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_graylog": "false", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_journald": "false", 2024-09-22T01:00:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_stderr": "false", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_syslog": "false", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max_rotating_auth_attempts": "10", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_action_on_write_error": "1", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_alternate_name_max": "8192", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_asio_thread_count": "2", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_export_pin": "true", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_dirs": "true", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_interval": "5", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-22T01:00:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_interval": "10", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_max": "-1", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_max_until": "-1", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_merge_size": "50", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_midchunk": "0.300000", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_min_start": "0.200000", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_minchunk": "0.001000", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_mode": "0", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_need_max": "1.200000", 2024-09-22T01:00:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_need_min": "0.800000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_overload_epochs": "2", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_bits": "3", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_size": "10000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_target_decay": "10.000000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_grace": "15.000000", 2024-09-22T01:00:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_interval": "4.000000", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_mid": "0.700000", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_delay": "0", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_release_free_interval": "10", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_reservation": "0.050000", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_interval": "1", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_threshold": "262144", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-22T01:00:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_prealloc_inos": "1000", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_connect_bootstrapping": "false", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_damage_table_max_entries": "10000", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_auth_pins": "false", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_frag": "false", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_scatterstat": "false", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_subtrees": "false", 2024-09-22T01:00:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_decay_halflife": "5.000000", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_default_dir_hash": "2", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_defer_session_stale": "true", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_deny_all_reconnect": "false", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_keys_per_op": "16384", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_max_commit_size": "10", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_max_entries": "0", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_prefetch": "true", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_on_map": "false", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_early_reply": "true", 2024-09-22T01:00:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_enable_op_tracker": "true", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_enforce_unique_name": "true", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_random": "true", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_health_summarize_threshold": "10", 2024-09-22T01:00:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_health_dummy": "false", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_migrator_session_race": "false", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_join_fs": "", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_journal_format": "1", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_create_at": "0", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_export_at": "0", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_import_at": "0", 2024-09-22T01:00:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_at": "0", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_expire_at": "0", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_replay_at": "0", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_link_at": "0", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_mdstable_at": "0", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_openc_at": "0", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_rename_at": "0", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_shutdown_at": "0", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_event_large_threshold": "524288", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_events_per_segment": "1024", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_max_events": "-1", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_max_segments": "128", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_pause": "false", 2024-09-22T01:00:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_segment_size": "0", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_trim_threshold": "128", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_warn_factor": "2.000000", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_caps_per_client": "1048576", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_completed_flushes": "100000", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_completed_requests": "100000", 2024-09-22T01:00:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_export_size": "20971520", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_file_recover": "32", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_files": "64", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_ops": "8192", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_snaps_per_dir": "100", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_metrics_update_interval": "2", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_min_caps_per_client": "100", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_min_caps_working_set": "10000", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_numa_node": "-1", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_complaint_time": "30.000000", 2024-09-22T01:00:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_duration": "600", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_size": "20", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_slow_op_size": "20", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_log_threshold": "5", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_ping_grace": "15", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_ping_interval": "5", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_caps": "30000", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_warning_threshold": "262144", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-22T01:00:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_replay_interval": "1.000000", 2024-09-22T01:00:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-22T01:00:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-22T01:00:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_root_ino_gid": "0", 2024-09-22T01:00:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_root_ino_uid": "0", 2024-09-22T01:00:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-22T01:00:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-22T01:00:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-22T01:00:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-22T01:00:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-22T01:00:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-22T01:00:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-22T01:00:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-22T01:00:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-22T01:00:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-22T01:00:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_shutdown_check": "0", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_skip_ino": "0", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_max_uid": "4294967294", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_min_uid": "0", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_rstat": "false", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_standby_replay_damaged": "false", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_symlink_recovery": "true", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_thrash_exports": "0", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_thrash_fragments": "0", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_tick_interval": "5.000000", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_valgrind_exit": "false", 2024-09-22T01:00:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_verify_backtrace": "1", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_verify_scatter": "false", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_wipe_sessions": "false", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool_debug": "false", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_device_bytes": "1073741824", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_page_set": "false", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_page_size": "65536", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_bytes": "134217728", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_messages": "512", 2024-09-22T01:00:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_disabled_modules": "", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_enable_op_tracker": "true", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_max_pg_num_change": "128", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mds_bytes": "134217728", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mds_messages": "128", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mon_bytes": "134217728", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mon_messages": "128", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-22T01:00:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_history_duration": "600", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_history_size": "20", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_log_threshold": "5", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_osd_bytes": "536870912", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_osd_messages": "8192", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_pool": "true", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_standby_modules": "true", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_stats_period": "5", 2024-09-22T01:00:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_stats_threshold": "5", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_tick_period": "2", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_allow_pool_delete": "true", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_allow_pool_size_one": "true", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_auth_validate_all_caps": "true", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_bytes": "104857600", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_directed_command_retry": "5", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-22T01:00:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_parallel": "3", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_log_interval": "1.000000", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_ping_interval": "10.000000", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_target_rank": "-1", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_file_level": "debug", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_file": "true", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-22T01:00:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_journald": "false", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_bootstrap": "false", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_start": "false", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_trim": "true", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cpu_threads": "4", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_crush_min_required_version": "hammer", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_daemon_bytes": "419430400", 2024-09-22T01:00:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_avail_crit": "5", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_avail_warn": "5", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_size_warn": "16106127360", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_json": "false", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_transactions": "false", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_extra_checks": "false", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-22T01:00:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_reef": "false", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_squid": "false", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_down_mkfs_grace": "120", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_down_uptime_grace": "60", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_election_default_strategy": "1", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_election_timeout": "5.000000", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ping_divisor": "2", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_enable_op_tracker": "true", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_fake_pool_delete": "false", 2024-09-22T01:00:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_force_quorum_join": "false", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_globalid_prealloc": "10000", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_detail_to_clog": "false", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_log_update_period": "5", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_max_detail": "50", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog": "true", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog_interval": "600", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_host": "172.21.15.2,[v2:172.21.15.2:3301,v1:172.21.15.2:6790],172.21.15.119", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_host_override": "", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_initial_members": "", 2024-09-22T01:00:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease": "5.000000", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_full_interval": "50", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_max": "10000", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_max_summary": "50", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_log_epochs": "500", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-22T01:00:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_osd": "10000", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_pg_per_osd": "10000", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_pool_pg_num": "65536", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_force_trim_to": "0", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_skip_sanity": "false", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_memory_autotune": "true", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_memory_target": "2147483648", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_beacon_grace": "30", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_digest_period": "5", 2024-09-22T01:00:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_inactive_grace": "60", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_min_osdmap_epochs": "500", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_complaint_time": "30", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_duration": "600", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_size": "20", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_slow_op_size": "20", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_log_threshold": "5", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_in": "false", 2024-09-22T01:00:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_cache_size": "500", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_down_out_interval": "600", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_force_trim_to": "0", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-22T01:00:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_down_reporters": "2", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_report_timeout": "900", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-22T01:00:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_stuck_threshold": "60", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-22T01:00:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_probe_timeout": "2.000000", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_max_change": "0.050000", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_max_osds": "4", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_interval": "86400", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_max_keys": "100", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_timeout": "300", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_session_timeout": "300", 2024-09-22T01:00:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_smart_report_timeout": "5", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stat_smooth_intervals": "6", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_pool_min_size": "2", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_pool_size": "4", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_debug": "false", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_provider_kill_at": "0", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_requester_kill_at": "0", 2024-09-22T01:00:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_timeout": "60.000000", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_target_pg_per_osd": "100", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_tick_interval": "5", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_timecheck_interval": "300.000000", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_use_min_delay_socket": "false", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_older_version_delay": "604800", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-22T01:00:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_misplaced": "false", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_older_version": "true", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-22T01:00:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "monmap": "", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_tracing_enabled": "false", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_op_threads": "3", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_cm": "false", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_device_name": "", 2024-09-22T01:00:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_dscp": "96", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_local_gid": "", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_port_num": "1", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_sl": "3", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_support_srq": "true", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_type": "ib", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_reap_threshold": "5", 2024-09-22T01:00:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_before_connect": "false", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_ipv4": "true", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_ipv6": "false", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_msgr1": "true", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_msgr2": "true", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_port_max": "7568", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_port_min": "6800", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_retry_count": "3", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_retry_delay": "5", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_client": "false", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mds": "false", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mgr": "false", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mon": "false", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_osd": "false", 2024-09-22T01:00:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_client_mode": "crc secure", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_cluster_mode": "crc secure", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_cluster_type": "", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_compress_secure": "false", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_connection_idle_timeout": "900", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_connection_ready_timeout": "10", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_crc_data": "true", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_crc_header": "true", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_bad_msg": "false", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_bug": "true", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_old_message": "true", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_skipped_message": "false", 2024-09-22T01:00:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_coremask": "0xF", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_enable_tso": "true", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hugepages": "", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_lro": "true", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_memory_channel": "4", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-22T01:00:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_pmd": "", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_port_id": "0", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dump_on_send": "false", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_initial_backoff": "0.200000", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_max": "1.000000", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_type": "", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_network_congestion": "0", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_socket_failures": "5000", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_learn_addr_from_peer": "true", 2024-09-22T01:00:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_max_accept_failures": "4", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_max_backoff": "15.000000", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_client_mode": "secure crc", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_service_mode": "secure crc", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compress_min_size": "1024", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compress_mode": "none", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_pq_min_cost": "65536", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_public_type": "", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_service_mode": "crc secure", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_listen_backlog": "512", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_nodelay": "true", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-22T01:00:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_rcvbuf": "0", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_type": "async+posix", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "no_config_file": "false", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_completion_locks_per_session": "32", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inflight_ops": "1024", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_tick_interval": "5.000000", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_timeout": "10.000000", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objectstore_blackhole": "false", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-22T01:00:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "openssl_engine_opts": "", 2024-09-22T01:00:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_delay_time": "5.000000", 2024-09-22T01:00:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_hist_halflife": "1000", 2024-09-22T01:00:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_max_low_ops": "2", 2024-09-22T01:00:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_max_ops": "4", 2024-09-22T01:00:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-22T01:00:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-22T01:00:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_slop": "0.020000", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_asio_thread_count": "2", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_async_recovery_min_cost": "100", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_scan_max": "512", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_scan_min": "64", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_degraded": "false", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_peering": "false", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_unfound": "true", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_beacon_report_interval": "300", 2024-09-22T01:00:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_duration": "30", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_max_block_size": "67108864", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_blkin_trace_all": "false", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_check_for_log_corruption": "false", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-22T01:00:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_update_on_start": "true", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_message_cap": "256", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_message_size_cap": "524288000", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_op_priority": "63", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_watch_timeout": "30", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_command_max_records": "256", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_compact_on_start": "false", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-22T01:00:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_update_on_start": "true", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_update_weight_set": "true", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_feed_pullee": "-1", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_misdirected_ops": "false", 2024-09-22T01:00:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_no_acting_change": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_no_purge_strays": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_op_order": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_shutdown": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_trim_objects": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_snaps": "false", 2024-09-22T01:00:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_keys": "1024", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_stride": "524288", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_default_notify_timeout": "30", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep": "0.000000", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-22T01:00:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_discard_disconnected_ops": "true", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_enable_op_tracker": "true", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_info": "true", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown": "true", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_function_tracing": "false", 2024-09-22T01:00:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_grace": "20", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_interval": "6", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_peers": "10", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_size": "2000", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_stale": "600", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_max_size": "100000", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_min_size": "1000", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-22T01:00:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal_size": "5120", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_kill_backfill_at": "0", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_cache_size": "50", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_dedup": "true", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_message_max": "40", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-22T01:00:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_share_max_epochs": "40", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_attr_name_len": "100", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_attr_size": "0", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_backfills": "1", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_markdown_count": "5", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_markdown_period": "600", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_name_len": "2048", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_namespace_len": "256", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_size": "134217728", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_blocked_by": "16", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_log_entries": "10000", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-22T01:00:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pgls": "1024", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_push_cost": "8388608", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_push_objects": "10", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_scrubs": "3", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_trimming_pgs": "2", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_write_op_reply_len": "64", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_write_size": "90", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-22T01:00:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_profile": "balanced", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-22T01:00:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_base": "805306368", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_cache_min": "134217728", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target": "4294967296", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target_autotune": "false", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_min_pg_log_entries": "250", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_min_recovery_priority": "0", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-22T01:00:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_report_interval": "5", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_num_cache_shards": "32", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_num_op_tracker_shard": "32", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_auto_affinity": "true", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_node": "-1", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_prefer_iface": "true", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objecter_finishers": "1", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore": "bluestore", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore_fuse": "false", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore_tracing": "false", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_complaint_time": "30.000000", 2024-09-22T01:00:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_duration": "600", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_size": "20", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_slow_op_size": "20", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_log_threshold": "5", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards": "0", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards_hdd": "5", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards_ssd": "8", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_pq_min_cost": "65536", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_queue_cut_off": "high", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-22T01:00:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_thread_timeout": "15", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_open_classes_on_start": "true", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_os_flags": "0", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_peering_op_priority": "255", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_delete_cost": "1048576", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_delete_priority": "5", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_trim_max": "10000", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_trim_min": "100", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-22T01:00:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_crimson": "false", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-22T01:00:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flags": "0", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_min_size": "0", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pg_num": "32", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pgp_num": "0", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_read_ratio": "70", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_size": "2", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_type": "replicated", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-22T01:00:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_push_per_object_cost": "1000", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recover_clone_overlap": "true", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_cost": "20971520", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active": "0", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-22T01:00:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_single_start": "1", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_op_priority": "3", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_priority": "5", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep": "0.000000", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_repair_during_recovery": "false", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_requested_scrub_priority": "120", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-22T01:00:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_auto_repair": "false", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_begin_hour": "0", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_begin_week_day": "0", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_chunk_max": "25", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_chunk_min": "5", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_cost": "52428800", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_during_recovery": "false", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_end_hour": "0", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_end_week_day": "0", 2024-09-22T01:00:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_event_cost": "4096", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_invalid_stats": "true", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_max_preemptions": "5", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_priority": "5", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_sleep": "0.000000", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-22T01:00:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_skip_data_digest": "false", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_smart_report_timeout": "5", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_cost": "1048576", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_priority": "5", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_target_transaction_size": "30", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-22T01:00:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tracing": "false", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_use_stale_snap": "false", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "osdc_blkin_trace_all": "false", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_kill_at": "0", 2024-09-22T01:00:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_max_join_drift": "10", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_min": "500", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_min_wait": "0.050000", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_propose_interval": "1.000000", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_max": "500", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_min": "250", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_stash_full_interval": "25", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_trim_max": "500", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_trim_min": "250", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "perf": "true", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-22T01:00:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "-", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrv": "", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "public_bind_addr": "-", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "public_network": "", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "public_network_interface": "", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "qat_compressor_busy_polling": "false", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "qat_compressor_enabled": "false", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "qat_compressor_session_max_number": "256", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_mon_op_timeout": "0", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_osd_op_timeout": "0", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_tracing": "false", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_atime_update_interval": "60", 2024-09-22T01:00:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_balance_parent_reads": "false", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_balance_snap_reads": "false", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blkin_trace_all": "false", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache": "true", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_policy": "writearound", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_size": "33554432", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-22T01:00:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_clone_copy_on_read": "false", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_compression_hint": "none", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_concurrent_management_ops": "10", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_clone_format": "auto", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_data_pool": "", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_features": "61", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_format": "2", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_map_options": "", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_order": "22", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_pool": "rbd", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-22T01:00:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_stripe_count": "0", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_stripe_unit": "0", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_enable_alloc_hint": "true", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_io_scheduler": "simple", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-22T01:00:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_order": "24", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_pool": "", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_splay_width": "4", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_localize_parent_reads": "false", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_localize_snap_reads": "false", 2024-09-22T01:00:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-22T01:00:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-22T01:00:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mtime_update_interval": "60", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_non_blocking_aio": "true", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_op_thread_timeout": "60", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_op_threads": "1", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_parent_cache_enabled": "false", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_plugins": "", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_burst": "0", 2024-09-22T01:00:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_limit": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_exclude_ops": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_burst": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_limit": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-22T01:00:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_read_from_replica_policy": "default", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_skip_partial_discard": "true", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_tracing": "false", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_validate_names": "true", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_validate_pool": "true", 2024-09-22T01:00:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "restapi_base_url": "", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "restapi_log_level": "", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_acl_grants_max_num": "100", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_admin_entry": "admin", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_backend_store": "rados", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_barbican_url": "", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_beast_enable_async": "true", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_counters_cache": "false", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-22T01:00:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_enabled": "true", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_expiry_interval": "900", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_lru_size": "25000", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_config_store": "rados", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_content_length_compat": "false", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_copy_obj_progress": "true", 2024-09-22T01:00:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cors_rules_max_num": "100", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cross_domain_policy": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_addr": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_password": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_username": "", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_require_ssl": "true", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-22T01:00:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-22T01:00:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_addr": "", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_auth": "token", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_namespace": "", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_prefix": "", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_token_file": "", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_buffersize": "524288", 2024-09-22T01:00:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_low_speed_time": "300", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d4n_port": "6379", 2024-09-22T01:00:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_changes_size": "1000", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_num_shards": "128", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_window": "30", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-22T01:00:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-22T01:00:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-22T01:00:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-22T01:00:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-22T01:00:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-22T01:00:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-22T01:00:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-22T01:00:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-22T01:00:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dns_name": "", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dns_s3website_name": "", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dynamic_resharding": "true", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-22T01:00:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_gc_threads": "true", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_lc_threads": "true", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_ops_log": "false", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_quota_threads": "true", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_static_website": "false", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_usage_log": "false", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enforce_swift_acls": "true", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_exit_timeout_secs": "120", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_expose_bucket": "false", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_extended_http_attrs": "", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_filter": "none", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_frontends": "beast port=7480", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_deferred": "50", 2024-09-22T01:00:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_objs": "32", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_processor_period": "3600", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_init_timeout": "300", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_kafka_connection_idle": "300", 2024-09-22T01:00:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_domain": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_password": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_password_path": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_project": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_tenant": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_token": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_token_path": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_user": "", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_api_version": "2", 2024-09-22T01:00:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_domain": "", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_password": "", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_project": "", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_user": "", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_url": "", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_debug_interval": "-1", 2024-09-22T01:00:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_lock_max_time": "90", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_objs": "32", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_rules": "1000", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_worker": "3", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_thread_delay": "0", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_dnattr": "uid", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_searchfilter": "", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_uri": "", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-22T01:00:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_http_headers": "", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_object_name_utc": "false", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attr_name_len": "0", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attr_size": "0", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_chunk_size": "4194304", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-22T01:00:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_listing_results": "1000", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_notify_retries": "10", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_put_param_size": "1048576", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_put_size": "5368709120", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_slo_entries": "1000", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_md_log_max_shards": "64", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_mp_lock_max_time": "600", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-22T01:00:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_max_gc": "300", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_num_async_rados_threads": "32", 2024-09-22T01:00:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_num_control_oids": "8", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_numa_node": "-1", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_chunk_size": "100", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_gc_interval": "600", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_thread_timeout": "600", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_tracing": "false", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_token": "", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_url": "", 2024-09-22T01:00:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_verify_ssl": "true", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_rados": "false", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_socket_path": "", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_push_interval": "2.000000", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_cache_lanes": "3", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-22T01:00:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_cache_partitions": "3", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_print_continue": "true", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_tracing": "false", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm": "", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm_id": "", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_region": "", 2024-09-22T01:00:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_request_uri": "", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_batch_size": "64", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_max_aio": "128", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_num_logs": "16", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_thread_interval": "600", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_resolve_cname": "false", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_run_sync_thread": "true", 2024-09-22T01:00:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_scheduler_type": "throttler", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_script_uri": "", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_service_provider_name": "", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_client_id": "", 2024-09-22T01:00:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_client_secret": "", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_entry": "sts", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_key": "sts", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_min_session_duration": "900", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_token_introspection_url": "", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_account_in_url": "false", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_auth_entry": "auth", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_auth_url": "", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_custom_header": "", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_need_stats": "true", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_tenant_name": "", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_token_expiration": "86400", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_url": "", 2024-09-22T01:00:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_url_prefix": "swift", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_lease_period": "120", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-22T01:00:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-22T01:00:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-22T01:00:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-22T01:00:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-22T01:00:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-22T01:00:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_thread_pool_size": "512", 2024-09-22T01:00:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-22T01:00:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-22T01:00:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_comment": "", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_createby": "", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_encoding": "", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_flag": "false", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_origin": "", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_tracker": "", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_trust_forwarded_https": "false", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_max_shards": "32", 2024-09-22T01:00:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_max_user_shards": "1", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_use_opa_authz": "false", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_counters_cache": "false", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_max_buckets": "1000", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_policies_max_num": "100", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_unique_email": "true", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_verify_ssl": "true", 2024-09-22T01:00:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone": "", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone_id": "", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup": "", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup_id": "", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rgwlc_auto_session_clear": "true", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_block_size": "4096", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-22T01:00:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_size": "536870912", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_index_type": "binary_search", 2024-09-22T01:00:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_partition_filters": "false", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_perf": "false", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "run_dir": "/var/run/ceph", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_block_create": "true", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_cache_lru_size": "67108864", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-22T01:00:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_default_max_object_size": "16777216", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_device_size": "53687091200", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_full_integrity_check": "false", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_capacity": "16", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_main_device_type": "SSD", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-22T01:00:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_segment_size": "67108864", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "set_keepcaps": "false", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "setgroup": "", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "setuser": "", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "setuser_match_path": "", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "thp": "false", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "threadpool_default_timeout": "60", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "throttler_perf_counter": "true" 2024-09-22T01:00:46.037 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:46.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-22T01:00:46.187 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a: {} 2024-09-22T01:00:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:00:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b: {} 2024-09-22T01:00:46.201 INFO:teuthology.orchestra.run.smithi002.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:00:46.210 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c: {} 2024-09-22T01:00:46.210 INFO:teuthology.orchestra.run.smithi002.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:00:46.226 DEBUG:teuthology.orchestra.run.smithi002:> type ceph-objectstore-tool 2024-09-22T01:00:46.230 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T01:00:46.231 DEBUG:teuthology.orchestra.run.smithi119:> type ceph-objectstore-tool 2024-09-22T01:00:46.236 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T01:00:46.237 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-22T01:00:46.248 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-22T01:00:46.248 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-22T01:00:46.248 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:00:46.248 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:00:46.249 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-22T01:00:46.249 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:00:46.249 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-22T01:00:46.249 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-22T01:00:46.250 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:00:46.250 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-22T01:00:46.250 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:00:46.250 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-22T01:00:46.250 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:00:46.251 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-22T01:00:46.274 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:46.272+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:46.293 INFO:teuthology.orchestra.run.smithi002.stdout:mapped writes DISABLED 2024-09-22T01:00:46.293 INFO:teuthology.orchestra.run.smithi002.stdout:mapped reads DISABLED 2024-09-22T01:00:46.293 INFO:teuthology.orchestra.run.smithi002.stdout:Seed set to 4604 2024-09-22T01:00:46.351 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:46.345+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:46.389 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:46.390 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:46.405 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:00:46.453 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:46.452+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:46.485 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:00:46.485 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:00:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:00:46.519 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:00:46.538 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:00:46.538 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:00:46.543 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:46.543 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:46.552 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:46.549+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:46.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:00:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:00:46.556 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:00:46.574 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:00:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:00:46.594 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:00:46.595 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:00:46.614 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:00:46.615 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:00:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:46.626 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 21, 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-22T01:00:18.015746+0000", 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-22T01:00:39.930825+0000", 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-22T01:00:30.876018+0000", 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-22T01:00:20.335765+0000", 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5799936, 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-22T01:00:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 5, 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 2, 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "jewel", 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-22T01:00:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:30.699559+0000", 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:00:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:00:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:00:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:00:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:00:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-22T01:00:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:36.410605+0000", 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:00:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-22T01:00:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:00:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 2, 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 21, 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:00:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:00:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9900000095367432, 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9900000095367432, 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "8c0f94a8-3c63-442d-9dc1-50c1278929da", 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 0, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:00:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2277223948", 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2277223948", 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2277223948", 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2277223948", 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "2d2cf30e-e9bd-40ba-8293-193faa02ea69", 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:00:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 18, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/463340194", 2024-09-22T01:00:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/463340194", 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/463340194", 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/463340194", 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a3d3e330-9ea1-4a5c-811b-f285f9a3cff4", 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:00:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 18, 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-22T01:00:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/387643397", 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/387643397", 2024-09-22T01:00:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/387643397", 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/387643397", 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "239072f0-4b69-4624-ba6c-733f299208da", 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:00:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 0, 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:00:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1215210764", 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1215210764", 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1215210764", 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1215210764", 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:00:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fd35cb24-eb2c-4c17-abe9-ce5c34db9014", 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 0, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:00:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6800", 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6801", 2024-09-22T01:00:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6802", 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6803", 2024-09-22T01:00:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6806", 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6807", 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6804", 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6805", 2024-09-22T01:00:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6801/2463696152", 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6803/2463696152", 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6807/2463696152", 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6805/2463696152", 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "b60a584d-5334-41e4-92d9-f6689de6b8bf", 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 18, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:00:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6824", 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1388509479 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6825", 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1388509479 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6826", 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1388509479 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6827", 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1388509479 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6830", 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1388509479 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6831", 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1388509479 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6828", 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1388509479 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6829", 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1388509479 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6825/1388509479", 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6827/1388509479", 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1388509479", 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1388509479", 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "9be24894-1100-4e4e-bf1b-79e163fd97e0", 2024-09-22T01:00:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 18, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6816", 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6817", 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:00:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6818", 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6819", 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6822", 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6823", 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6820", 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6821", 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6817/4220217922", 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6819/4220217922", 2024-09-22T01:00:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6823/4220217922", 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6821/4220217922", 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "293cfbc3-8c2e-40b5-ab89-0a124cc8651c", 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:00:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 18, 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6808", 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6809", 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6810", 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:00:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6811", 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6814", 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:00:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6815", 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6812", 2024-09-22T01:00:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6813", 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6809/3867156106", 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6811/3867156106", 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3867156106", 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3867156106", 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:00:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:00:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.556128+0000", 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.159753+0000", 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:00:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.766199+0000", 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:00:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.155529+0000", 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:00:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.226232+0000", 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.243346+0000", 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.143084+0000", 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:00:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.524369+0000", 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-22T01:00:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [ 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 2, 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 1 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [ 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 2, 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 1 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:00:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-22T01:00:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:46.698 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:46.698 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:00:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:00:46.700 INFO:tasks.thrashosds.thrasher:j is {'epoch': 21, 'fsid': '7d08b428-4185-4f0f-a0fc-f7dc1fefdce9', 'created': '2024-09-22T01:00:18.015746+0000', 'modified': '2024-09-22T01:00:39.930825+0000', 'last_up_change': '2024-09-22T01:00:30.876018+0000', 'last_in_change': '2024-09-22T01:00:20.335765+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:00:30.699559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:00:36.410605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8c0f94a8-3c63-442d-9dc1-50c1278929da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2277223948}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2277223948}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2277223948}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2277223948}]}, 'public_addr': '172.21.15.2:6825/2277223948', 'cluster_addr': '172.21.15.2:6827/2277223948', 'heartbeat_back_addr': '172.21.15.2:6831/2277223948', 'heartbeat_front_addr': '172.21.15.2:6829/2277223948', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 463340194}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 463340194}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 463340194}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 463340194}]}, 'public_addr': '172.21.15.2:6817/463340194', 'cluster_addr': '172.21.15.2:6819/463340194', 'heartbeat_back_addr': '172.21.15.2:6823/463340194', 'heartbeat_front_addr': '172.21.15.2:6821/463340194', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 387643397}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 387643397}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 387643397}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 387643397}]}, 'public_addr': '172.21.15.2:6801/387643397', 'cluster_addr': '172.21.15.2:6803/387643397', 'heartbeat_back_addr': '172.21.15.2:6807/387643397', 'heartbeat_front_addr': '172.21.15.2:6805/387643397', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '239072f0-4b69-4624-ba6c-733f299208da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1215210764}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1215210764}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1215210764}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1215210764}]}, 'public_addr': '172.21.15.2:6809/1215210764', 'cluster_addr': '172.21.15.2:6811/1215210764', 'heartbeat_back_addr': '172.21.15.2:6815/1215210764', 'heartbeat_front_addr': '172.21.15.2:6813/1215210764', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 2463696152}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 2463696152}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 2463696152}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 2463696152}]}, 'public_addr': '172.21.15.119:6801/2463696152', 'cluster_addr': '172.21.15.119:6803/2463696152', 'heartbeat_back_addr': '172.21.15.119:6807/2463696152', 'heartbeat_front_addr': '172.21.15.119:6805/2463696152', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b60a584d-5334-41e4-92d9-f6689de6b8bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1388509479}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1388509479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1388509479}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1388509479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1388509479}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1388509479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1388509479}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1388509479}]}, 'public_addr': '172.21.15.119:6825/1388509479', 'cluster_addr': '172.21.15.119:6827/1388509479', 'heartbeat_back_addr': '172.21.15.119:6831/1388509479', 'heartbeat_front_addr': '172.21.15.119:6829/1388509479', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9be24894-1100-4e4e-bf1b-79e163fd97e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 4220217922}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 4220217922}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 4220217922}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 4220217922}]}, 'public_addr': '172.21.15.119:6817/4220217922', 'cluster_addr': '172.21.15.119:6819/4220217922', 'heartbeat_back_addr': '172.21.15.119:6823/4220217922', 'heartbeat_front_addr': '172.21.15.119:6821/4220217922', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '293cfbc3-8c2e-40b5-ab89-0a124cc8651c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3867156106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3867156106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3867156106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3867156106}]}, 'public_addr': '172.21.15.119:6809/3867156106', 'cluster_addr': '172.21.15.119:6811/3867156106', 'heartbeat_back_addr': '172.21.15.119:6815/3867156106', 'heartbeat_front_addr': '172.21.15.119:6813/3867156106', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.556128+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.159753+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.766199+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.155529+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.226232+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.243346+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.143084+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.524369+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:00:46.701 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:00:46.702 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:46.702 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:46.702 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:46.696+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:46.703 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:00:46.799 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:46.796+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:46.854 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:46.854 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:46.868 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:00:46.900 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:46.897+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:46.971 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:00:47.001 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:46.997+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:47.026 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:47.026 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:47.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:00:47.055 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:00:47.056 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:00:47.073 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-22T01:00:46.968819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":248184,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726536,"statfs":{"total":773094113280,"available":772839972864,"internally_reserved":0,"allocated":2867200,"data_stored":1752110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.033114"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.193083+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:00:40.193083+0000","last_peered":"2024-09-22T01:00:40.193083+0000","last_clean":"2024-09-22T01:00:40.193083+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:00:40.193083+0000","last_undegraded":"2024-09-22T01:00:40.193083+0000","last_fullsized":"2024-09-22T01:00:40.193083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946521+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:00:39.946521+0000","last_peered":"2024-09-22T01:00:39.946521+0000","last_clean":"2024-09-22T01:00:39.946521+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:00:39.946521+0000","last_undegraded":"2024-09-22T01:00:39.946521+0000","last_fullsized":"2024-09-22T01:00:39.946521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354336+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:00:40.354336+0000","last_peered":"2024-09-22T01:00:40.354336+0000","last_clean":"2024-09-22T01:00:40.354336+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:00:40.354336+0000","last_undegraded":"2024-09-22T01:00:40.354336+0000","last_fullsized":"2024-09-22T01:00:40.354336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.946358+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:00:39.946358+0000","last_peered":"2024-09-22T01:00:39.946358+0000","last_clean":"2024-09-22T01:00:39.946358+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:00:39.946358+0000","last_undegraded":"2024-09-22T01:00:39.946358+0000","last_fullsized":"2024-09-22T01:00:39.946358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.951945+0000","last_change":"2024-09-22T01:00:39.947537+0000","last_active":"2024-09-22T01:00:39.951945+0000","last_peered":"2024-09-22T01:00:39.951945+0000","last_clean":"2024-09-22T01:00:39.951945+0000","last_became_active":"2024-09-22T01:00:37.928490+0000","last_became_peered":"2024-09-22T01:00:37.928490+0000","last_unstale":"2024-09-22T01:00:39.951945+0000","last_undegraded":"2024-09-22T01:00:39.951945+0000","last_fullsized":"2024-09-22T01:00:39.951945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:36.384733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.945210+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:00:39.945210+0000","last_peered":"2024-09-22T01:00:39.945210+0000","last_clean":"2024-09-22T01:00:39.945210+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:00:39.945210+0000","last_undegraded":"2024-09-22T01:00:39.945210+0000","last_fullsized":"2024-09-22T01:00:39.945210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.354460+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:00:40.354460+0000","last_peered":"2024-09-22T01:00:40.354460+0000","last_clean":"2024-09-22T01:00:40.354460+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:00:40.354460+0000","last_undegraded":"2024-09-22T01:00:40.354460+0000","last_fullsized":"2024-09-22T01:00:40.354460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:39.947793+0000","last_change":"2024-09-22T01:00:39.947793+0000","last_active":"2024-09-22T01:00:39.947793+0000","last_peered":"2024-09-22T01:00:39.947793+0000","last_clean":"2024-09-22T01:00:39.947793+0000","last_became_active":"2024-09-22T01:00:37.927774+0000","last_became_peered":"2024-09-22T01:00:37.927774+0000","last_unstale":"2024-09-22T01:00:39.947793+0000","last_undegraded":"2024-09-22T01:00:39.947793+0000","last_fullsized":"2024-09-22T01:00:39.947793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:33.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088854699999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-22T01:00:40.353465+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:00:40.353465+0000","last_peered":"2024-09-22T01:00:40.353465+0000","last_clean":"2024-09-22T01:00:40.353465+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:00:40.353465+0000","last_undegraded":"2024-09-22T01:00:40.353465+0000","last_fullsized":"2024-09-22T01:00:40.353465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T01:00:46.960642+0000","last_change":"2024-09-22T01:00:46.960642+0000","last_active":"2024-09-22T01:00:46.960642+0000","last_peered":"2024-09-22T01:00:46.960642+0000","last_clean":"2024-09-22T01:00:46.960642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T01:00:46.960642+0000","last_undegraded":"2024-09-22T01:00:46.960642+0000","last_fullsized":"2024-09-22T01:00:46.960642+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":802816,"data_stored":661785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:00:47.075 INFO:tasks.thrashosds.thrasher:No pools; doing nothing 2024-09-22T01:00:47.102 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:47.101+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:47.165 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:47.165 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:47.177 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:00:47.203 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:47.197+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:47.290 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:47.291 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:47.304 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:00:47.304 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:47.300+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:47.404 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:47.401+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:47.463 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:47.463 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:47.476 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:00:47.505 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:47.501+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:47.596 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:47.596 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:47.605 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:47.601+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:47.608 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:00:47.706 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:47.704+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:47.741 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:47.742 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:47.755 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:00:47.807 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:47.805+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:47.908 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:47.904+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:47.918 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:47.918 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:47.933 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:00:48.009 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:48.005+0000 7f629e482640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26643) UID: 0 2024-09-22T01:00:48.106 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:48.106 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:48.111 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:48.109+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:48.122 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:00:48.212 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:48.212+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:48.255 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:48.255 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:48.268 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:00:48.313 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:48.312+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:48.387 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:48.387 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:48.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:00:48.414 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:48.412+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:48.514 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:48.509+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:48.519 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:48.519 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:48.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:00:48.616 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:48.612+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:48.699 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:48.699 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:48.702 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:00:48.715 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:00:48.749 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:48.748+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:48.817 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:48.812+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:48.837 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:48.837 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:48.852 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:00:48.861 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:00:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:00:48.878 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:00:48.878 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:00:48.897 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:00:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:00:48.915 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:00:48.915 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:00:48.918 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:48.916+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:48.936 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:00:48.936 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:00:48.956 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:00:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:00:48.977 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:48.977 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:00:48.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:00:48.977 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:37.926040+0000", 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.052353002, 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082686770000000003, 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4235", 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/2507802733", 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:00:48.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.926040+0000", 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.926040+0000", 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:48.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.926049+0000", 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5329999999999994e-06 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.926065+0000", 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6350000000000001e-05 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.926099+0000", 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3543000000000001e-05 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.926122+0000", 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3247e-05 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.926287+0000", 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000164919 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.926310+0000", 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3019000000000001e-05 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.928564+0000", 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022545130000000001 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.928960+0000", 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039554 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.929284+0000", 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00032411400000000002 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932414+0000", 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031297 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.934044+0000", 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001630177 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.934181+0000", 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013772900000000001 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.934309+0000", 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000127293 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:48.984 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4235.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:37.932047+0000", 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.046346472, 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037907099999999998, 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:48.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932047+0000", 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932047+0000", 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932049+0000", 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9269999999999999e-06 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932061+0000", 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2306999999999999e-05 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932100+0000", 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9183000000000001e-05 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932123+0000", 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2840000000000002e-05 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932389+0000", 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026637500000000001 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932402+0000", 2024-09-22T01:00:48.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3029e-05 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.932426+0000", 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3410000000000001e-05 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:37.939001+0000", 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.039392132, 2024-09-22T01:00:48.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040731699999999998, 2024-09-22T01:00:48.990 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:48.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:48.990 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:00:48.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:48.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4235", 2024-09-22T01:00:48.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/2507802733", 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.939001+0000", 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:48.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.939001+0000", 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.939004+0000", 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.52e-06 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.939015+0000", 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1749999999999999e-05 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:48.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.939044+0000", 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.866e-05 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.939066+0000", 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2099999999999998e-05 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.939180+0000", 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114152 2024-09-22T01:00:48.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.939408+0000", 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000228135 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4235.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:37.940043+0000", 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.038350007, 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000355688, 2024-09-22T01:00:48.994 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4235", 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/2507802733", 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.940043+0000", 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.940043+0000", 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.940045+0000", 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8700000000000001e-06 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.940055+0000", 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6169999999999996e-06 2024-09-22T01:00:48.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.940079+0000", 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4266000000000001e-05 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.940091+0000", 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2514000000000001e-05 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.940203+0000", 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011202200000000001 2024-09-22T01:00:48.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:37.940399+0000", 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019539899999999999 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4235.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:38.937696+0000", 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.040696695999999, 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084242989999999997, 2024-09-22T01:00:48.998 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4235", 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/2507802733", 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.937696+0000", 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:48.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.937696+0000", 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.937701+0000", 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3000000000000003e-06 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.937721+0000", 2024-09-22T01:00:49.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0400000000000001e-05 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.937765+0000", 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3797000000000001e-05 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.937789+0000", 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3723e-05 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:49.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.937917+0000", 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012794600000000001 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.937927+0000", 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0309000000000001e-05 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.943089+0000", 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005162374 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.943314+0000", 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022507899999999999 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.943527+0000", 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000212301 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945019+0000", 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001492342 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945920+0000", 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00090119899999999999 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.946004+0000", 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3917000000000003e-05 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.946121+0000", 2024-09-22T01:00:49.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116612 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4235.0:5 2.2 e20/18 ondisk, result = 0)", 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:38.945725+0000", 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.032667656999999, 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00041017000000000001, 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945725+0000", 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945725+0000", 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945728+0000", 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-06 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945738+0000", 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0027000000000001e-05 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945767+0000", 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9532000000000001e-05 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945783+0000", 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5693e-05 2024-09-22T01:00:49.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945894+0000", 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011120600000000001 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.945909+0000", 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4837e-05 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:38.946136+0000", 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022667499999999999 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4235.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:39.947440+0000", 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0309527240000005, 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046237179999999998, 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4235", 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/2507802733", 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.947440+0000", 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.947440+0000", 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.947444+0000", 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2729999999999998e-06 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.947462+0000", 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082999999999999e-05 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.947497+0000", 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4977000000000001e-05 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.947522+0000", 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5286000000000002e-05 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.947705+0000", 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018303199999999999 2024-09-22T01:00:49.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.947938+0000", 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023336799999999999 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.948338+0000", 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039940100000000002 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950904+0000", 2024-09-22T01:00:49.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002566657 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.951785+0000", 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000880859 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.951941+0000", 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015586899999999999 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:49.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.952064+0000", 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012291299999999999 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4235.0:7 2.2 e21/18 ondisk, result = 0)", 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:39.950675+0000", 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0277182049999993, 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025238799999999998, 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950675+0000", 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950675+0000", 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950685+0000", 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9629999999999993e-06 2024-09-22T01:00:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950694+0000", 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2730000000000008e-06 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950723+0000", 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9283000000000001e-05 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950737+0000", 2024-09-22T01:00:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3943e-05 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950858+0000", 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120559 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950885+0000", 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7407e-05 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:39.950927+0000", 2024-09-22T01:00:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1959999999999998e-05 2024-09-22T01:00:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:49.019 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:49.019 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:00:49.020 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:00:49.021 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:00:49.021 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:00:49.022 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:00:49.022 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:49.017+0000 7f629e482640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26643) UID: 0 2024-09-22T01:00:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:00:49.122 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:49.117+0000 7f629e482640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26643) UID: 0 2024-09-22T01:00:49.188 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:00:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:00:49.189 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:49.204 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:00:49.223 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:49.220+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:49.323 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:49.320+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:49.371 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:00:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:00:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:00:49.371 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:49.385 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:00:49.424 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:49.420+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:49.503 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:49.503 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:00:49.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:00:49.503 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-22T01:00:49.503 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:49.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:00:49.525 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:49.524+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:49.625 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:49.621+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:49.641 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:49.642 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:00:49.642 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:00:49.642 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:49.656 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:00:49.703 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:00:49.726 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:49.724+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:49.825 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:49.825 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:00:49.826 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:00:49.826 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:00:49.826 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:49.827 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:49.821+0000 7f629e482640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26643) UID: 0 2024-09-22T01:00:49.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:00:49.928 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:49.924+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:50.014 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:50.014 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:00:50.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:00:50.014 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:00:50.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.119859975000001, 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:00:50.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:00:50.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.560481+0000", 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.450594121999998, 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011266162999999999, 2024-09-22T01:00:50.021 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 35 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560481+0000", 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560481+0000", 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560481+0000", 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4700000000000001e-07 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560573+0000", 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1872000000000004e-05 2024-09-22T01:00:50.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560591+0000", 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7272999999999999e-05 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560599+0000", 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.567674+0000", 2024-09-22T01:00:50.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007075072 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.567809+0000", 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000134919 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.568020+0000", 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021110200000000001 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.571391+0000", 2024-09-22T01:00:50.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003370811 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.571497+0000", 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105772 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.571636+0000", 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013921300000000001 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.571747+0000", 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011076199999999999 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.560616+0000", 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.450459286000001, 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011422768, 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560616+0000", 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560616+0000", 2024-09-22T01:00:50.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560616+0000", 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5300000000000003e-07 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560638+0000", 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-05 2024-09-22T01:00:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560649+0000", 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1673e-05 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560657+0000", 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8630000000000008e-06 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.569021+0000", 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083641619999999996 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.569179+0000", 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000157183 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.569418+0000", 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000239714 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.571798+0000", 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002379632 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.571899+0000", 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100809 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.571980+0000", 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1242999999999994e-05 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.572038+0000", 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8266000000000001e-05 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.560671+0000", 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.450403797, 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011983554, 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560671+0000", 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560671+0000", 2024-09-22T01:00:50.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560672+0000", 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2699999999999999e-07 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560678+0000", 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560688+0000", 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.022e-05 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.560697+0000", 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.570327+0000", 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096308120000000007 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.570466+0000", 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013835599999999999 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.570716+0000", 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025027099999999999 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.572078+0000", 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001361936 2024-09-22T01:00:50.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.572453+0000", 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037560399999999998 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.572547+0000", 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3893000000000004e-05 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.572655+0000", 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107222 2024-09-22T01:00:50.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.657590+0000", 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.353484809000001, 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011427909, 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.039 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 88 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.657590+0000", 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.657590+0000", 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.657591+0000", 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3570000000000001e-06 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.657603+0000", 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1357e-05 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.657638+0000", 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5095999999999997e-05 2024-09-22T01:00:50.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.657653+0000", 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.522e-05 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.666004+0000", 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083506459999999998 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.666255+0000", 2024-09-22T01:00:50.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025087800000000001 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.666650+0000", 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039575400000000001 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.668199+0000", 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001548581 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.668867+0000", 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00066833799999999998 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.668954+0000", 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7065999999999996e-05 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.669018+0000", 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3615999999999995e-05 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.824972+0000", 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.186103148000001, 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01284772, 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 109 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.045 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.824972+0000", 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.824972+0000", 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.824973+0000", 2024-09-22T01:00:50.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.663e-06 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825130+0000", 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000156885 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825158+0000", 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7606999999999998e-05 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825176+0000", 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8462999999999999e-05 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.830856+0000", 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056792359999999998 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.830993+0000", 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137346 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.831210+0000", 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000217011 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.834657+0000", 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034475230000000001 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.837675+0000", 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030172440000000001 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.837755+0000", 2024-09-22T01:00:50.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0222000000000005e-05 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.837819+0000", 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4519999999999999e-05 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.825212+0000", 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.185862419999999, 2024-09-22T01:00:50.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012749666999999999, 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 110 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825212+0000", 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825212+0000", 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825214+0000", 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.72e-06 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825227+0000", 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2366e-05 2024-09-22T01:00:50.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825253+0000", 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6213000000000001e-05 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825270+0000", 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7280000000000001e-05 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.832229+0000", 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069589730000000002 2024-09-22T01:00:50.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.832394+0000", 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000165456 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.832591+0000", 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019694100000000001 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.834967+0000", 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023754589999999999 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.837835+0000", 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028679640000000002 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.837905+0000", 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9949000000000004e-05 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.837962+0000", 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7346000000000003e-05 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.825300+0000", 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.185775296999999, 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012882483, 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.056 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 111 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825300+0000", 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825300+0000", 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825301+0000", 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825314+0000", 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3023999999999999e-05 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825335+0000", 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0718999999999999e-05 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825353+0000", 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8114e-05 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.833768+0000", 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084154610000000008 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.833899+0000", 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013012500000000001 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.834096+0000", 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019733199999999999 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.837653+0000", 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035575659999999999 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.837983+0000", 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00032996700000000002 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.838124+0000", 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014038899999999999 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.838182+0000", 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8199999999999998e-05 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.061 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.825420+0000", 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.185654381999999, 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013988135, 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 112 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825420+0000", 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825420+0000", 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825422+0000", 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.79e-06 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825616+0000", 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019397899999999999 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825661+0000", 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4623000000000003e-05 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825679+0000", 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7782999999999999e-05 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.835098+0000", 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094197039999999992 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.835339+0000", 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024104399999999999 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.835617+0000", 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027724200000000003 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.838342+0000", 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027256979999999999 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.839271+0000", 2024-09-22T01:00:50.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00092822599999999999 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.839347+0000", 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6777e-05 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.839409+0000", 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1268999999999999e-05 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.825712+0000", 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.185362420000001, 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015427982999999999, 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.068 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 113 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825712+0000", 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825712+0000", 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825714+0000", 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2559999999999999e-06 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825723+0000", 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4700000000000008e-06 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825736+0000", 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2957e-05 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825745+0000", 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9630000000000004e-06 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.836575+0000", 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010830045 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.836719+0000", 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000144093 2024-09-22T01:00:50.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.836931+0000", 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021192800000000001 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.840992+0000", 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040607719999999998 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.841012+0000", 2024-09-22T01:00:50.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0163e-05 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.841081+0000", 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9302999999999997e-05 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.841140+0000", 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9033000000000002e-05 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.825760+0000", 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.185315191000001, 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016735280000000002, 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 114 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.074 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825760+0000", 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825760+0000", 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825760+0000", 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2600000000000002e-07 2024-09-22T01:00:50.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825766+0000", 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3000000000000001e-06 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825776+0000", 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0427e-05 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825786+0000", 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0000000000000001e-05 2024-09-22T01:00:50.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.838430+0000", 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012643728999999999 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.838563+0000", 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013322200000000001 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.838746+0000", 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018248799999999999 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.841277+0000", 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025314140000000001 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.842360+0000", 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010835249999999999 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.842433+0000", 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2940000000000003e-05 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.842495+0000", 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1508999999999997e-05 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.825806+0000", 2024-09-22T01:00:50.079 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.185268388000001, 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017733226000000001, 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 115 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825806+0000", 2024-09-22T01:00:50.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825806+0000", 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825807+0000", 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9299999999999997e-07 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825957+0000", 2024-09-22T01:00:50.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014963899999999999 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.825972+0000", 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.484e-05 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826009+0000", 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7456000000000002e-05 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.839540+0000", 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013530624999999999 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.839758+0000", 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021800500000000001 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.839967+0000", 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020908499999999999 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.843387+0000", 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034196869999999998 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.843412+0000", 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5446999999999999e-05 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.843480+0000", 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7682999999999996e-05 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.843540+0000", 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9966000000000002e-05 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.826030+0000", 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.185044943000001, 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018181585, 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 116 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826030+0000", 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826030+0000", 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826031+0000", 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.04e-06 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826040+0000", 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4429999999999992e-06 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826052+0000", 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2097e-05 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826061+0000", 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5429999999999994e-06 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.841386+0000", 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015324701 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.841548+0000", 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016273200000000001 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.841797+0000", 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024849099999999999 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.843629+0000", 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001832487 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.844083+0000", 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000453826 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.844159+0000", 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5459999999999999e-05 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.844211+0000", 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2765e-05 2024-09-22T01:00:50.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:32.826074+0000", 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.185001190000001, 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018700998, 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 117 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826074+0000", 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826074+0000", 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826074+0000", 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9699999999999995e-07 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826082+0000", 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5000000000000002e-06 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826092+0000", 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7999999999999993e-06 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.826099+0000", 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6199999999999999e-06 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.842586+0000", 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016486812 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.842753+0000", 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000166971 2024-09-22T01:00:50.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.842955+0000", 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020179200000000001 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.844345+0000", 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001390393 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.844644+0000", 2024-09-22T01:00:50.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029825400000000002 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.844721+0000", 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7145999999999996e-05 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.844775+0000", 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4012999999999999e-05 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.096 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:33.327813+0000", 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.683261609999999, 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011212041000000001, 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 140 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.097 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327813+0000", 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327813+0000", 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327814+0000", 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327931+0000", 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117143 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327948+0000", 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7410000000000001e-05 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327959+0000", 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0662999999999999e-05 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.335782+0000", 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078226600000000004 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.335901+0000", 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011927 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.336094+0000", 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000192709 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.337667+0000", 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015728929999999999 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.338969+0000", 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013028200000000001 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.339000+0000", 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0412e-05 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.339025+0000", 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5401e-05 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:33.327975+0000", 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.683099513999998, 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011811416999999999, 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 141 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327975+0000", 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327975+0000", 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327976+0000", 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4300000000000002e-07 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327982+0000", 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7440000000000004e-06 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.327994+0000", 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2013e-05 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.328018+0000", 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3906000000000001e-05 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.336807+0000", 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087894889999999993 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.336874+0000", 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6230999999999997e-05 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.336962+0000", 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8134999999999996e-05 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.337747+0000", 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00078528199999999999 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.339720+0000", 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019726819999999999 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.339759+0000", 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9029999999999997e-05 2024-09-22T01:00:50.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:33.339787+0000", 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8062e-05 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:34.258441+0000", 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.752633635, 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011585471, 2024-09-22T01:00:50.108 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 156 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258441+0000", 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258441+0000", 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258442+0000", 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258448+0000", 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:00:50.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258459+0000", 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0849999999999999e-05 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258470+0000", 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1263e-05 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.264205+0000", 2024-09-22T01:00:50.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057347989999999996 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.264345+0000", 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013987599999999999 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.264549+0000", 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000203962 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.266757+0000", 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022084159999999999 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.269856+0000", 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030986659999999999 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.269931+0000", 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5508999999999998e-05 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.270027+0000", 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5279999999999996e-05 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:34.258483+0000", 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.752591559000001, 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012705472000000001, 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.114 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 157 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258483+0000", 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258483+0000", 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258484+0000", 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0999999999999998e-07 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258490+0000", 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3269999999999997e-06 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258501+0000", 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.045e-05 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258511+0000", 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716e-05 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.265244+0000", 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067320189999999997 2024-09-22T01:00:50.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.265382+0000", 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013864199999999999 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.265582+0000", 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019998800000000001 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.270111+0000", 2024-09-22T01:00:50.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045286290000000002 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.271041+0000", 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000929859 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.271118+0000", 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7498999999999998e-05 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.271189+0000", 2024-09-22T01:00:50.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0632999999999999e-05 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:34.258524+0000", 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.752551109000001, 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015366458, 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.120 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 158 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258524+0000", 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258524+0000", 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258524+0000", 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258626+0000", 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101993 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258639+0000", 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.252e-05 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258659+0000", 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9703000000000001e-05 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.266972+0000", 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083135529999999996 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.267105+0000", 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013247199999999999 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.267313+0000", 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020811500000000001 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.270199+0000", 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028859010000000002 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.273688+0000", 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034895859999999998 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.273818+0000", 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129536 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.273890+0000", 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2336000000000007e-05 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.125 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:34.258673+0000", 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.752401534000001, 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015381511, 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:00:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 159 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258673+0000", 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258673+0000", 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258674+0000", 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5000000000000001e-07 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258683+0000", 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5969999999999992e-06 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258693+0000", 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0183000000000001e-05 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.258701+0000", 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5669999999999997e-06 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.268796+0000", 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010095119 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.268936+0000", 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140455 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.269134+0000", 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000197779 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.270620+0000", 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014860679999999999 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.273905+0000", 2024-09-22T01:00:50.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032854580000000002 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.273970+0000", 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4672999999999996e-05 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:34.274055+0000", 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4761999999999999e-05 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:50.131 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:50.133 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:00:50.133 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:50.132+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:50.233 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:50.229+0000 7f629e482640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26643) UID: 0 2024-09-22T01:00:50.270 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:50.270 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:50.270 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:50.270 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:50.285 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:00:50.334 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:50.332+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:50.408 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:50.409 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:00:50.409 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:00:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:50.422 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:00:50.465 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:50.464+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:50.536 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:50.533+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:50.588 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.602201+0000", 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.984066686999999, 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025993399999999999, 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602201+0000", 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602201+0000", 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602203+0000", 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.39e-06 2024-09-22T01:00:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602216+0000", 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3053e-05 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602235+0000", 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9487000000000002e-05 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602246+0000", 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0530000000000001e-05 2024-09-22T01:00:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602386+0000", 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014039799999999999 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602427+0000", 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1170000000000001e-05 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.604729+0000", 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023015850000000001 2024-09-22T01:00:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.604800+0000", 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1727e-05 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.603445+0000", 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.982822977000001, 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025514370000000001, 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603445+0000", 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603445+0000", 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603446+0000", 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1230000000000001e-06 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603624+0000", 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017823599999999999 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603657+0000", 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2435999999999999e-05 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603675+0000", 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8199999999999999e-05 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603812+0000", 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137665 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603865+0000", 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2370000000000002e-05 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.605923+0000", 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020578240000000002 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.605996+0000", 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3583000000000003e-05 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.629078+0000", 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.957190185000002, 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025968499999999999, 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629078+0000", 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629078+0000", 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629079+0000", 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6840000000000001e-06 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629091+0000", 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.147e-05 2024-09-22T01:00:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629137+0000", 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6131999999999998e-05 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629152+0000", 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.508e-05 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629277+0000", 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012528300000000001 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629340+0000", 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3169000000000005e-05 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.631552+0000", 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022115730000000001 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.631674+0000", 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122459 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.630415+0000", 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.955853109, 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024280249999999999, 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630415+0000", 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630415+0000", 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630416+0000", 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.26e-06 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630467+0000", 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1490000000000003e-05 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630497+0000", 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9609999999999999e-05 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630511+0000", 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630637+0000", 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000126279 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630671+0000", 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4316000000000001e-05 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.632735+0000", 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020630940000000001 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.632843+0000", 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108193 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.658190+0000", 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.928077518999999, 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025791159999999998, 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658190+0000", 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658190+0000", 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658192+0000", 2024-09-22T01:00:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6899999999999999e-06 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658203+0000", 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1163e-05 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658232+0000", 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9269999999999999e-05 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658251+0000", 2024-09-22T01:00:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8595999999999999e-05 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658428+0000", 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017739200000000001 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658478+0000", 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0083000000000001e-05 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.660668+0000", 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021899900000000002 2024-09-22T01:00:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.660769+0000", 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100932 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:87 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'54)", 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.664768+0000", 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.921499659999998, 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025929500000000001, 2024-09-22T01:00:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664768+0000", 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664768+0000", 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664770+0000", 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664813+0000", 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3056999999999998e-05 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664837+0000", 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4349e-05 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664851+0000", 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4157e-05 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664996+0000", 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014512499999999999 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.665034+0000", 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7737000000000001e-05 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.667284+0000", 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022499590000000002 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.667361+0000", 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7132999999999995e-05 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:131 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-22T01:00:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.328153+0000", 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.258114797000001, 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002762352, 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328153+0000", 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328153+0000", 2024-09-22T01:00:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328155+0000", 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328288+0000", 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133509 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328314+0000", 2024-09-22T01:00:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5437000000000001e-05 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328330+0000", 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6189999999999999e-05 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328417+0000", 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6948999999999997e-05 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328447+0000", 2024-09-22T01:00:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.029e-05 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330843+0000", 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023956670000000002 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330915+0000", 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2490000000000006e-05 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:132 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.328950+0000", 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.25731828, 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003008227, 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328950+0000", 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328950+0000", 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328951+0000", 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.46e-06 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328963+0000", 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2454e-05 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328989+0000", 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.603e-05 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.329006+0000", 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6616000000000002e-05 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.329559+0000", 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000552759 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.329617+0000", 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8433000000000001e-05 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331858+0000", 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022411089999999998 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331958+0000", 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9365999999999997e-05 2024-09-22T01:00:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'87)", 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.330658+0000", 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.255609925999998, 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031151149999999999, 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330658+0000", 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330658+0000", 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330659+0000", 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4300000000000001e-06 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330826+0000", 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016705499999999999 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330853+0000", 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6659999999999999e-05 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330872+0000", 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9326000000000001e-05 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330965+0000", 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2528999999999994e-05 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330997+0000", 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1624000000000001e-05 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333705+0000", 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002708322 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333773+0000", 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8169000000000004e-05 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:135 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.331292+0000", 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.254975910999999, 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025748400000000001, 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331292+0000", 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331292+0000", 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331294+0000", 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5999999999999999e-06 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331306+0000", 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2207e-05 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331341+0000", 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4986e-05 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331361+0000", 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0205999999999999e-05 2024-09-22T01:00:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332086+0000", 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072495100000000005 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332149+0000", 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3232999999999996e-05 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333780+0000", 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016304640000000001 2024-09-22T01:00:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333867+0000", 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7193000000000004e-05 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.331985+0000", 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.25428329, 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036613750000000001, 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331985+0000", 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331985+0000", 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331987+0000", 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2230000000000001e-06 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331999+0000", 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1936999999999999e-05 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332032+0000", 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2855999999999998e-05 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332055+0000", 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3349999999999998e-05 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332904+0000", 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084926299999999995 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332979+0000", 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4560000000000004e-05 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335581+0000", 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026027530000000002 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335646+0000", 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4432999999999998e-05 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:137 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'90)", 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.333230+0000", 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.253037786, 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003227162, 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333230+0000", 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333230+0000", 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333232+0000", 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333459+0000", 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022676099999999999 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333488+0000", 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8937000000000001e-05 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333511+0000", 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3255999999999999e-05 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333939+0000", 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00042832699999999999 2024-09-22T01:00:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333985+0000", 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6159999999999999e-05 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336394+0000", 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002408828 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336457+0000", 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3072999999999997e-05 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:138 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'90)", 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.334015+0000", 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.252252951999999, 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035801259999999999, 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334015+0000", 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334015+0000", 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334016+0000", 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.373e-06 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334027+0000", 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0896e-05 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334060+0000", 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2787000000000003e-05 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334074+0000", 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3709999999999999e-05 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334980+0000", 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090614600000000003 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335011+0000", 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.154e-05 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337512+0000", 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025007639999999999 2024-09-22T01:00:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337595+0000", 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2910000000000004e-05 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'90)", 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.334784+0000", 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.251484241, 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029124070000000001, 2024-09-22T01:00:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334784+0000", 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334784+0000", 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334785+0000", 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.186e-06 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334799+0000", 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3886999999999999e-05 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334820+0000", 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1005999999999998e-05 2024-09-22T01:00:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334830+0000", 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0063e-05 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335704+0000", 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087400399999999995 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335736+0000", 2024-09-22T01:00:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2505999999999997e-05 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337610+0000", 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001873506 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337696+0000", 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6248999999999994e-05 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'92)", 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.335939+0000", 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.250329199999999, 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025217210000000002, 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335939+0000", 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335939+0000", 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335940+0000", 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3599999999999999e-06 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336058+0000", 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118069 2024-09-22T01:00:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336082+0000", 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3745999999999999e-05 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336094+0000", 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336525+0000", 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00043157999999999998 2024-09-22T01:00:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336563+0000", 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7413000000000002e-05 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.338396+0000", 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018334359999999999 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.338460+0000", 2024-09-22T01:00:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'92)", 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.336689+0000", 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.249578486000001, 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024972869999999999, 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336689+0000", 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336689+0000", 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336691+0000", 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2699999999999999e-06 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336699+0000", 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336719+0000", 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9513000000000001e-05 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336729+0000", 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0727000000000001e-05 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337760+0000", 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010305519999999999 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337798+0000", 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8445999999999997e-05 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.339098+0000", 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012996939999999999 2024-09-22T01:00:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.339187+0000", 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8465000000000001e-05 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:142 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'100, mlcod=16'98)", 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.340482+0000", 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.245785549000001, 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035520859999999999, 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340482+0000", 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340482+0000", 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340484+0000", 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9099999999999999e-06 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340575+0000", 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1026000000000006e-05 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340605+0000", 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9839999999999999e-05 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340619+0000", 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3637e-05 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340755+0000", 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013614500000000001 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340791+0000", 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6514000000000003e-05 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.343931+0000", 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003139518 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.344034+0000", 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103496 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:143 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-22T01:00:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.341197+0000", 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.245070721000001, 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029417810000000001, 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341197+0000", 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341197+0000", 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341199+0000", 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.407e-06 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341273+0000", 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3815999999999999e-05 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341301+0000", 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7457000000000002e-05 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341318+0000", 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7598999999999999e-05 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341557+0000", 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023837199999999999 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341593+0000", 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6712999999999999e-05 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.344048+0000", 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024540809999999999 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.344139+0000", 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1335999999999994e-05 2024-09-22T01:00:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:158 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'108)", 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:34.267115+0000", 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.319152991999999, 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024160340000000001, 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267115+0000", 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267115+0000", 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267116+0000", 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0049999999999999e-06 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267207+0000", 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0818999999999999e-05 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267225+0000", 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7897000000000002e-05 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267233+0000", 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9929999999999995e-06 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267331+0000", 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8713000000000002e-05 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267371+0000", 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9409999999999997e-05 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.269472+0000", 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021015919999999998 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.269531+0000", 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8604999999999999e-05 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:49.726480+0000", 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85978790900000002, 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055351139999999998, 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726480+0000", 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726480+0000", 2024-09-22T01:00:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726482+0000", 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.097e-06 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726493+0000", 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.113e-05 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726514+0000", 2024-09-22T01:00:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0647999999999999e-05 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726525+0000", 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1018e-05 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726640+0000", 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114806 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op must wait for map", 2024-09-22T01:00:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726659+0000", 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9598000000000001e-05 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.729402+0000", 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027432020000000001 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.729645+0000", 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000242365 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-22T01:00:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.729996+0000", 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035108600000000002 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.731444+0000", 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014483510000000001 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.731923+0000", 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047893799999999999 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.731977+0000", 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4293999999999999e-05 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.732015+0000", 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7580999999999998e-05 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:50.676 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:50.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:00:50.678 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:50.676+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:50.779 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:50.773+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:50.844 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:50.844 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:50.844 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:50.844 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:50.858 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:00:50.880 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:50.877+0000 7f629e482640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26643) UID: 0 2024-09-22T01:00:50.972 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:50.972 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:50.973 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:00:50.973 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:00:50.973 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:50.981 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:50.977+0000 7f629e482640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26643) UID: 0 2024-09-22T01:00:50.985 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:00:51.024 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:00:51.083 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:51.081+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4235.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:38.943514+0000", 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.171203218, 2024-09-22T01:00:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016454880000000001, 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943514+0000", 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943514+0000", 2024-09-22T01:00:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943515+0000", 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943527+0000", 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2333e-05 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943569+0000", 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1610000000000003e-05 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943597+0000", 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7773999999999999e-05 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943746+0000", 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000149765 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943801+0000", 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4623000000000002e-05 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.945052+0000", 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012512599999999999 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.945160+0000", 2024-09-22T01:00:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107336 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4235.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:39.948331+0000", 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.166386763, 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017906630000000001, 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948331+0000", 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948331+0000", 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948332+0000", 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3239999999999999e-06 2024-09-22T01:00:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948356+0000", 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4156e-05 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948386+0000", 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0334000000000001e-05 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948411+0000", 2024-09-22T01:00:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948650+0000", 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000238951 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948713+0000", 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2498999999999999e-05 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.950035+0000", 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001321904 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.950121+0000", 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6621999999999998e-05 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9 3.1a e25/22 3:5cea7035:::rbd_info:head v 25'2, mlcod=0'0)", 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:49.716660+0000", 2024-09-22T01:00:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.3980572490000001, 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00109387, 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716660+0000", 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716660+0000", 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716660+0000", 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7399999999999999e-07 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716670+0000", 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4120000000000007e-06 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716697+0000", 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7608999999999999e-05 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716709+0000", 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1384e-05 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716792+0000", 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3599999999999999e-05 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716825+0000", 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2391000000000002e-05 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.717703+0000", 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087801399999999999 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.717754+0000", 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1186000000000001e-05 2024-09-22T01:00:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:51.128 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:51.129 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:00:51.165 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:00:51.165 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:00:51.183 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:51.181+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:51.193 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:00:51.194 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:00:51.211 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:00:51.211 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:00:51.227 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:00:51.227 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:00:51.248 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:00:51.248 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:00:51.253 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:51.253 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:51.266 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:00:51.268 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:00:51.269 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:00:51.289 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:00:51.289 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:00:51.289 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:51.288+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:51.310 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:00:51.310 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:00:51.379 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:51.379 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:51.385 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:51.381+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:51.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:00:51.487 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:51.484+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:51.520 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:51.520 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:51.536 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:00:51.588 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:51.585+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:51.674 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0x23b7d71 2024-09-22T01:00:51.676 INFO:teuthology.orchestra.run.smithi002.stdout:1 trunc from 0x0 to 0x23b7d71 2024-09-22T01:00:51.689 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:51.685+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:51.703 INFO:teuthology.orchestra.run.smithi002.stdout:2 writesame 0x87c0515 thru 0x87c3a8e data_size 0xa(0x357a bytes) 2024-09-22T01:00:51.703 INFO:teuthology.orchestra.run.smithi002.stdout:3 read 0x1d108b1 thru 0x1d14c8b (0x43db bytes) 2024-09-22T01:00:51.703 INFO:teuthology.orchestra.run.smithi002.stdout:5 write 0x7d95e6d thru 0x7da0e14 (0xafa8 bytes) 2024-09-22T01:00:51.710 INFO:teuthology.orchestra.run.smithi002.stdout:6 write 0xdad1c6 thru 0xdb531b (0x8156 bytes) 2024-09-22T01:00:51.713 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:51.714 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:51.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:00:51.734 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:00:51.790 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:51.785+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:51.843 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:51.843 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:51.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:00:51.890 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:00:51.885+0000 7f629e482640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26643) UID: 0 2024-09-22T01:00:51.921 INFO:teuthology.orchestra.run.smithi002.stdout:7 read 0x3c0d880 thru 0x3c12d13 (0x5494 bytes) 2024-09-22T01:00:51.921 INFO:teuthology.orchestra.run.smithi002.stdout:9 read 0x37656a0 thru 0x376fd90 (0xa6f1 bytes) 2024-09-22T01:00:51.921 INFO:teuthology.orchestra.run.smithi002.stdout:10 writesame 0x1f09da1 thru 0x1f12faa data_size 0xba(0x920a bytes) 2024-09-22T01:00:51.926 INFO:teuthology.orchestra.run.smithi002.stdout:12 read 0x739c9d thru 0x744869 (0xabcd bytes) 2024-09-22T01:00:51.927 INFO:teuthology.orchestra.run.smithi002.stdout:13 read 0x4d95960 thru 0x4da34a2 (0xdb43 bytes) 2024-09-22T01:00:51.927 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:51.927 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:51.927 INFO:teuthology.orchestra.run.smithi002.stdout:18 writesame 0xfbb4e0 thru 0xfc5933 data_size 0x1a(0xa454 bytes) 2024-09-22T01:00:51.932 INFO:teuthology.orchestra.run.smithi002.stdout:19 read 0x58965e5 thru 0x589e586 (0x7fa2 bytes) 2024-09-22T01:00:51.932 INFO:teuthology.orchestra.run.smithi002.stdout:23 punch from 0x7ad92e7 to 0x7ae3b5f, (0xa878 bytes) 2024-09-22T01:00:51.932 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0x7431a7f 2024-09-22T01:00:51.933 INFO:teuthology.orchestra.run.smithi002.stdout:24 trunc from 0x87c3a8f to 0x7431a7f 2024-09-22T01:00:51.951 INFO:teuthology.orchestra.run.smithi002.stdout:25 write 0x55f6577 thru 0x55fb743 (0x51cd bytes) 2024-09-22T01:00:51.958 INFO:teuthology.orchestra.run.smithi002.stdout:28 read 0x59ca04f thru 0x59d1fc5 (0x7f77 bytes) 2024-09-22T01:00:51.971 INFO:teuthology.orchestra.run.smithi002.stdout:29 write 0x83cbd90 thru 0x83d8a2f (0xcca0 bytes) 2024-09-22T01:00:51.977 INFO:teuthology.orchestra.run.smithi002.stdout:30 write 0x4bd7582 thru 0x4be5767 (0xe1e6 bytes) 2024-09-22T01:00:51.984 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:51.984 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:51.984 INFO:teuthology.orchestra.run.smithi002.stdout:33 read 0x3355775 thru 0x3360a95 (0xb321 bytes) 2024-09-22T01:00:51.984 INFO:teuthology.orchestra.run.smithi002.stdout:34 writesame 0x546cc32 thru 0x546d308 data_size 0x1(0x6d7 bytes) 2024-09-22T01:00:51.988 INFO:teuthology.orchestra.run.smithi002.stdout:35 read 0x5d2d282 thru 0x5d34440 (0x71bf bytes) 2024-09-22T01:00:51.991 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:51.988+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:51.996 INFO:teuthology.orchestra.run.smithi002.stdout:36 trunc from 0x83d8a30 to 0x36e1240 2024-09-22T01:00:52.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:00:52.016 INFO:teuthology.orchestra.run.smithi002.stdout:38 write 0xb84247 thru 0xb8dc11 (0x99cb bytes) 2024-09-22T01:00:52.021 INFO:teuthology.orchestra.run.smithi002.stdout:41 read 0x2f4e3d4 thru 0x2f58954 (0xa581 bytes) 2024-09-22T01:00:52.046 INFO:teuthology.orchestra.run.smithi002.stdout:42 writesame 0xcbd28cb thru 0xcbd56d2 data_size 0xc(0x2e08 bytes) 2024-09-22T01:00:52.046 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:52.046 INFO:teuthology.orchestra.run.smithi002.stdout:46 write 0x775d0a9 thru 0x7769585 (0xc4dd bytes) 2024-09-22T01:00:52.052 INFO:teuthology.orchestra.run.smithi002.stdout:47 punch from 0x91079b4 to 0x9108095, (0x6e1 bytes) 2024-09-22T01:00:52.052 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xdf9aba0 2024-09-22T01:00:52.054 INFO:teuthology.orchestra.run.smithi002.stdout:48 trunc from 0xcbd56d3 to 0xdf9aba0 2024-09-22T01:00:52.064 INFO:teuthology.orchestra.run.smithi002.stdout:50 write 0xaec71a3 thru 0xaece66a (0x74c8 bytes) 2024-09-22T01:00:52.070 INFO:teuthology.orchestra.run.smithi002.stdout:54 write 0x1a36faf thru 0x1a3ac9f (0x3cf1 bytes) 2024-09-22T01:00:52.076 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:00:52.076 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:00:52.076 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:00:52.076 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-22T01:00:52.076 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:00:52.077 INFO:teuthology.orchestra.run.smithi002.stdout:55 punch from 0xd302779 to 0xd30b267, (0x8aee bytes) 2024-09-22T01:00:52.077 INFO:teuthology.orchestra.run.smithi002.stdout:57 read 0xd010527 thru 0xd01538c (0x4e66 bytes) 2024-09-22T01:00:52.078 INFO:teuthology.orchestra.run.smithi002.stdout:59 trunc from 0xdf9aba0 to 0xd3d4e2d 2024-09-22T01:00:52.092 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:52.088+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:52.108 INFO:teuthology.orchestra.run.smithi002.stdout:62 trunc from 0xd3d4e2d to 0x19ac25a 2024-09-22T01:00:52.136 INFO:teuthology.orchestra.run.smithi002.stdout:63 write 0x225bf18 thru 0x2260e41 (0x4f2a bytes) 2024-09-22T01:00:52.138 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:52.139 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:52.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:00:52.157 INFO:tasks.ceph.osd.5:Stopped 2024-09-22T01:00:52.157 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-22T01:00:52.157 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T01:00:52.158 DEBUG:teuthology.orchestra.run.smithi119:> type ceph-objectstore-tool 2024-09-22T01:00:52.162 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T01:00:52.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-22T01:00:52.177 INFO:teuthology.orchestra.run.smithi002.stdout:65 write 0xdbd9022 thru 0xdbe7961 (0xe940 bytes) 2024-09-22T01:00:52.191 INFO:teuthology.orchestra.run.smithi002.stdout:67 write 0xdc21964 thru 0xdc29304 (0x79a1 bytes) 2024-09-22T01:00:52.192 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:52.188+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:52.197 INFO:teuthology.orchestra.run.smithi002.stdout:69 writesame 0xbc46b6c thru 0xbc47781 data_size 0xe(0xc16 bytes) 2024-09-22T01:00:52.203 INFO:teuthology.orchestra.run.smithi002.stdout:71 punch from 0x3709f61 to 0x3716073, (0xc112 bytes) 2024-09-22T01:00:52.227 INFO:teuthology.orchestra.run.smithi002.stdout:74 trunc from 0xdc29305 to 0x32e4c02 2024-09-22T01:00:52.247 INFO:teuthology.orchestra.run.smithi002.stdout:75 punch from 0x1c75679 to 0x1c7b1d6, (0x5b5d bytes) 2024-09-22T01:00:52.277 INFO:teuthology.orchestra.run.smithi002.stdout:76 write 0x9e92e02 thru 0x9e957d6 (0x29d5 bytes) 2024-09-22T01:00:52.284 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:52.284 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:52.292 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:52.289+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:52.293 INFO:teuthology.orchestra.run.smithi002.stdout:77 trunc from 0x9e957d7 to 0x66c5347 2024-09-22T01:00:52.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:00:52.313 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:52.313 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:52.313 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xe58c3ea 2024-09-22T01:00:52.333 INFO:teuthology.orchestra.run.smithi002.stdout:81 trunc from 0x66c5347 to 0xe58c3ea 2024-09-22T01:00:52.351 INFO:teuthology.orchestra.run.smithi002.stdout:82 read 0x1bd38d thru 0x1c4636 (0x72aa bytes) 2024-09-22T01:00:52.351 INFO:teuthology.orchestra.run.smithi002.stdout:84 write 0x88f50f9 thru 0x88fd663 (0x856b bytes) 2024-09-22T01:00:52.357 INFO:teuthology.orchestra.run.smithi002.stdout:86 writesame 0x914f684 thru 0x915d9d8 data_size 0x13(0xe355 bytes) 2024-09-22T01:00:52.366 INFO:teuthology.orchestra.run.smithi002.stdout:87 read 0xab8796a thru 0xab96850 (0xeee7 bytes) 2024-09-22T01:00:52.366 INFO:teuthology.orchestra.run.smithi002.stdout:88 writesame 0x2ac20ea thru 0x2acb5c4 data_size 0x1(0x94db bytes) 2024-09-22T01:00:52.393 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:52.389+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:52.420 INFO:teuthology.orchestra.run.smithi002.stdout:89 write 0x4b93c65 thru 0x4b966c2 (0x2a5e bytes) 2024-09-22T01:00:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:90 write 0x3d628e2 thru 0x3d6b6ad (0x8dcc bytes) 2024-09-22T01:00:52.433 INFO:teuthology.orchestra.run.smithi002.stdout:91 write 0x62f1a9f thru 0x62ff748 (0xdcaa bytes) 2024-09-22T01:00:52.468 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:52.468 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:52.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:00:52.494 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:52.492+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:52.595 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:52.589+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:52.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:00:52.704 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:52.701+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:52.735 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:00:52.797 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:52.793+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:52.829 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:52.829 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:52.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:00:52.898 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:52.896+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:52.997 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:52.997 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:52.998 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:52.993+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:53.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:00:53.100 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:53.096+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:53.183 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:53.183 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:53.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:00:53.248 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:53.245+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:53.301 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:53.297+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:53.317 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:53.317 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:53.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:00:53.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:00:53.402 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:53.400+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:53.485 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:00:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:00:53.493 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:53.493 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:53.502 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:00:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:00:53.503 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:53.500+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:53.507 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:00:53.521 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:00:53.522 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:00:53.538 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:00:53.538 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:00:53.556 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:00:53.557 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:00:53.557 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:00:53.577 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:00:53.577 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:00:53.596 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:00:53.596 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:00:53.599 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:00:53.600 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:00:53.604 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:53.601+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:53.625 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:53.625 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:53.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:00:53.725 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:53.724+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:53.743 INFO:teuthology.orchestra.run.smithi002.stdout:92 write 0x70dd30f thru 0x70e2d89 (0x5a7b bytes) 2024-09-22T01:00:53.748 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:53.749 INFO:teuthology.orchestra.run.smithi002.stdout:94 read 0x2e680b9 thru 0x2e6d6a2 (0x55ea bytes) 2024-09-22T01:00:53.749 INFO:teuthology.orchestra.run.smithi002.stdout:95 write 0x624a2f thru 0x62a9c5 (0x5f97 bytes) 2024-09-22T01:00:53.753 INFO:teuthology.orchestra.run.smithi002.stdout:96 writesame 0x767ff75 thru 0x76863e6 data_size 0x2e(0x6472 bytes) 2024-09-22T01:00:53.758 INFO:teuthology.orchestra.run.smithi002.stdout:97 read 0xa13a21a thru 0xa140b70 (0x6957 bytes) 2024-09-22T01:00:53.758 INFO:teuthology.orchestra.run.smithi002.stdout:99 write 0x7a4472e thru 0x7a5301b (0xe8ee bytes) 2024-09-22T01:00:53.762 INFO:teuthology.orchestra.run.smithi002.stdout:100 write 0xc03cbc9 thru 0xc042408 (0x5840 bytes) 2024-09-22T01:00:53.767 INFO:teuthology.orchestra.run.smithi002.stdout:101 write 0x48fedc9 thru 0x490d7e9 (0xea21 bytes) 2024-09-22T01:00:53.770 INFO:teuthology.orchestra.run.smithi002.stdout:102 write 0xd242a90 thru 0xd24fe44 (0xd3b5 bytes) 2024-09-22T01:00:53.775 INFO:teuthology.orchestra.run.smithi002.stdout:103 read 0xceb89ea thru 0xcebb310 (0x2927 bytes) 2024-09-22T01:00:53.775 INFO:teuthology.orchestra.run.smithi002.stdout:104 writesame 0x7418df3 thru 0x7422247 data_size 0x17(0x9455 bytes) 2024-09-22T01:00:53.778 INFO:teuthology.orchestra.run.smithi002.stdout:106 write 0x3656a55 thru 0x365f39f (0x894b bytes) 2024-09-22T01:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout:108 read 0x7882c42 thru 0x7891195 (0xe554 bytes) 2024-09-22T01:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout:110 clone 1 order 21 su 8192 sc 11 2024-09-22T01:00:53.805 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:53.804+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:53.809 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:53.809 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:53.809 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:00:53.809 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:00:53.809 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:53.824 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:00:53.906 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:53.901+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:53.993 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:00:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:00:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.602201+0000", 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.389150568000002, 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025993399999999999, 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602201+0000", 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602201+0000", 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602203+0000", 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.39e-06 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602216+0000", 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3053e-05 2024-09-22T01:00:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602235+0000", 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9487000000000002e-05 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602246+0000", 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0530000000000001e-05 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602386+0000", 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014039799999999999 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.602427+0000", 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1170000000000001e-05 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.604729+0000", 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023015850000000001 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.604800+0000", 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1727e-05 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.603445+0000", 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.387906858000001, 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025514370000000001, 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603445+0000", 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603445+0000", 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603446+0000", 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1230000000000001e-06 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603624+0000", 2024-09-22T01:00:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017823599999999999 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603657+0000", 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2435999999999999e-05 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603675+0000", 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8199999999999999e-05 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603812+0000", 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137665 2024-09-22T01:00:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.603865+0000", 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2370000000000002e-05 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.605923+0000", 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020578240000000002 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.605996+0000", 2024-09-22T01:00:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3583000000000003e-05 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.629078+0000", 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.362274066000001, 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025968499999999999, 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629078+0000", 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629078+0000", 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629079+0000", 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6840000000000001e-06 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629091+0000", 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.147e-05 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629137+0000", 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6131999999999998e-05 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629152+0000", 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.508e-05 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629277+0000", 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012528300000000001 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.629340+0000", 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3169000000000005e-05 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.631552+0000", 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022115730000000001 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.631674+0000", 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122459 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.630415+0000", 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.360936989999999, 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024280249999999999, 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630415+0000", 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630415+0000", 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630416+0000", 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.26e-06 2024-09-22T01:00:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630467+0000", 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1490000000000003e-05 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630497+0000", 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9609999999999999e-05 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630511+0000", 2024-09-22T01:00:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630637+0000", 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000126279 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.630671+0000", 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4316000000000001e-05 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.632735+0000", 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020630940000000001 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.632843+0000", 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108193 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.658190+0000", 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.333161400000002, 2024-09-22T01:00:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025791159999999998, 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658190+0000", 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658190+0000", 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658192+0000", 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6899999999999999e-06 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658203+0000", 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1163e-05 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658232+0000", 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9269999999999999e-05 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658251+0000", 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8595999999999999e-05 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658428+0000", 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017739200000000001 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.658478+0000", 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0083000000000001e-05 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.660668+0000", 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021899900000000002 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.660769+0000", 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100932 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:87 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'54)", 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:32.664768+0000", 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.326583541000002, 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025929500000000001, 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664768+0000", 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664768+0000", 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664770+0000", 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664813+0000", 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3056999999999998e-05 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664837+0000", 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4349e-05 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664851+0000", 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4157e-05 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.664996+0000", 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014512499999999999 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.665034+0000", 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7737000000000001e-05 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.667284+0000", 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022499590000000002 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:32.667361+0000", 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7132999999999995e-05 2024-09-22T01:00:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:131 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.328153+0000", 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.663198678000001, 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002762352, 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328153+0000", 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328153+0000", 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328155+0000", 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328288+0000", 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133509 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328314+0000", 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5437000000000001e-05 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328330+0000", 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6189999999999999e-05 2024-09-22T01:00:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328417+0000", 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6948999999999997e-05 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328447+0000", 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.029e-05 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330843+0000", 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023956670000000002 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330915+0000", 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2490000000000006e-05 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:132 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.328950+0000", 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.662402160999999, 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003008227, 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328950+0000", 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328950+0000", 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328951+0000", 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.46e-06 2024-09-22T01:00:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328963+0000", 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2454e-05 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.328989+0000", 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.603e-05 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.329006+0000", 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6616000000000002e-05 2024-09-22T01:00:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.329559+0000", 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000552759 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.329617+0000", 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8433000000000001e-05 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331858+0000", 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022411089999999998 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331958+0000", 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9365999999999997e-05 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'87)", 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.330658+0000", 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.660693807000001, 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031151149999999999, 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330658+0000", 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330658+0000", 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330659+0000", 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4300000000000001e-06 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330826+0000", 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016705499999999999 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330853+0000", 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6659999999999999e-05 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330872+0000", 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9326000000000001e-05 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330965+0000", 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2528999999999994e-05 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.330997+0000", 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1624000000000001e-05 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333705+0000", 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002708322 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333773+0000", 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8169000000000004e-05 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:135 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.331292+0000", 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.660059791999998, 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025748400000000001, 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331292+0000", 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331292+0000", 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331294+0000", 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5999999999999999e-06 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331306+0000", 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2207e-05 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331341+0000", 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4986e-05 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331361+0000", 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0205999999999999e-05 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332086+0000", 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072495100000000005 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332149+0000", 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3232999999999996e-05 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333780+0000", 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016304640000000001 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333867+0000", 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7193000000000004e-05 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.331985+0000", 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.659367171, 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036613750000000001, 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331985+0000", 2024-09-22T01:00:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331985+0000", 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331987+0000", 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2230000000000001e-06 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.331999+0000", 2024-09-22T01:00:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1936999999999999e-05 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332032+0000", 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2855999999999998e-05 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332055+0000", 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3349999999999998e-05 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332904+0000", 2024-09-22T01:00:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084926299999999995 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.332979+0000", 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4560000000000004e-05 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335581+0000", 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026027530000000002 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335646+0000", 2024-09-22T01:00:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4432999999999998e-05 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:137 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'90)", 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.333230+0000", 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.658121667, 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003227162, 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333230+0000", 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333230+0000", 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333232+0000", 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333459+0000", 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022676099999999999 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333488+0000", 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8937000000000001e-05 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333511+0000", 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3255999999999999e-05 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333939+0000", 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00042832699999999999 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.333985+0000", 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6159999999999999e-05 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336394+0000", 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002408828 2024-09-22T01:00:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336457+0000", 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3072999999999997e-05 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:138 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'90)", 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.334015+0000", 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.657336832999999, 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035801259999999999, 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334015+0000", 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334015+0000", 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334016+0000", 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.373e-06 2024-09-22T01:00:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334027+0000", 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0896e-05 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334060+0000", 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2787000000000003e-05 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334074+0000", 2024-09-22T01:00:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3709999999999999e-05 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334980+0000", 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090614600000000003 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335011+0000", 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.154e-05 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337512+0000", 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025007639999999999 2024-09-22T01:00:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337595+0000", 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2910000000000004e-05 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'90)", 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.334784+0000", 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.656568121999999, 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029124070000000001, 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334784+0000", 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334784+0000", 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334785+0000", 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.186e-06 2024-09-22T01:00:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334799+0000", 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3886999999999999e-05 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334820+0000", 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1005999999999998e-05 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.334830+0000", 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0063e-05 2024-09-22T01:00:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335704+0000", 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087400399999999995 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335736+0000", 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2505999999999997e-05 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337610+0000", 2024-09-22T01:00:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001873506 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337696+0000", 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6248999999999994e-05 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'92)", 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.335939+0000", 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.655413080999999, 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025217210000000002, 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335939+0000", 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335939+0000", 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.335940+0000", 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3599999999999999e-06 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336058+0000", 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118069 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336082+0000", 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3745999999999999e-05 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336094+0000", 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336525+0000", 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00043157999999999998 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336563+0000", 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7413000000000002e-05 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.338396+0000", 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018334359999999999 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.338460+0000", 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'92)", 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.336689+0000", 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.654662367, 2024-09-22T01:00:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024972869999999999, 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336689+0000", 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336689+0000", 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336691+0000", 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2699999999999999e-06 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336699+0000", 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336719+0000", 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9513000000000001e-05 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.336729+0000", 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0727000000000001e-05 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337760+0000", 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010305519999999999 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.337798+0000", 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8445999999999997e-05 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.339098+0000", 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012996939999999999 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.339187+0000", 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8465000000000001e-05 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:142 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'100, mlcod=16'98)", 2024-09-22T01:00:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.340482+0000", 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.65086943, 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035520859999999999, 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340482+0000", 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340482+0000", 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340484+0000", 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9099999999999999e-06 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340575+0000", 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1026000000000006e-05 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340605+0000", 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9839999999999999e-05 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340619+0000", 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3637e-05 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340755+0000", 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013614500000000001 2024-09-22T01:00:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.340791+0000", 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6514000000000003e-05 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.343931+0000", 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003139518 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.344034+0000", 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103496 2024-09-22T01:00:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:143 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:33.341197+0000", 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.650154602000001, 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029417810000000001, 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341197+0000", 2024-09-22T01:00:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341197+0000", 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341199+0000", 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.407e-06 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341273+0000", 2024-09-22T01:00:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3815999999999999e-05 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341301+0000", 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7457000000000002e-05 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341318+0000", 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7598999999999999e-05 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341557+0000", 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023837199999999999 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.341593+0000", 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6712999999999999e-05 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.344048+0000", 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024540809999999999 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:33.344139+0000", 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1335999999999994e-05 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4198.0:158 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'108)", 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:34.267115+0000", 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.724236872999999, 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024160340000000001, 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267115+0000", 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267115+0000", 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267116+0000", 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0049999999999999e-06 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267207+0000", 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0818999999999999e-05 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267225+0000", 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7897000000000002e-05 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267233+0000", 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9929999999999995e-06 2024-09-22T01:00:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267331+0000", 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8713000000000002e-05 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.267371+0000", 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9409999999999997e-05 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.269472+0000", 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021015919999999998 2024-09-22T01:00:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:34.269531+0000", 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8604999999999999e-05 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:49.726480+0000", 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2648717899999999, 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055351139999999998, 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726480+0000", 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726480+0000", 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726482+0000", 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.097e-06 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726493+0000", 2024-09-22T01:00:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.113e-05 2024-09-22T01:00:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726514+0000", 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0647999999999999e-05 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726525+0000", 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1018e-05 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726640+0000", 2024-09-22T01:00:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114806 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op must wait for map", 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.726659+0000", 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9598000000000001e-05 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.729402+0000", 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027432020000000001 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.729645+0000", 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000242365 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.729996+0000", 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035108600000000002 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.731444+0000", 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014483510000000001 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.731923+0000", 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047893799999999999 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.731977+0000", 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4293999999999999e-05 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.732015+0000", 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7580999999999998e-05 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.082 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.082 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:54.083 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:00:54.083 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:54.077+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:54.151 INFO:teuthology.orchestra.run.smithi119.stdout:3.d 2024-09-22T01:00:54.151 INFO:teuthology.orchestra.run.smithi119.stdout:3.c 2024-09-22T01:00:54.151 INFO:teuthology.orchestra.run.smithi119.stdout:3.1f 2024-09-22T01:00:54.151 INFO:teuthology.orchestra.run.smithi119.stdout:3.1c 2024-09-22T01:00:54.151 INFO:teuthology.orchestra.run.smithi119.stdout:3.17 2024-09-22T01:00:54.151 INFO:teuthology.orchestra.run.smithi119.stdout:3.16 2024-09-22T01:00:54.151 INFO:teuthology.orchestra.run.smithi119.stdout:3.5 2024-09-22T01:00:54.152 INFO:teuthology.orchestra.run.smithi119.stdout:3.1d 2024-09-22T01:00:54.152 INFO:teuthology.orchestra.run.smithi119.stdout:2.3 2024-09-22T01:00:54.152 INFO:teuthology.orchestra.run.smithi119.stdout:3.2 2024-09-22T01:00:54.152 INFO:teuthology.orchestra.run.smithi119.stdout:2.2 2024-09-22T01:00:54.184 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:54.181+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:54.212 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:54.213 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:54.213 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:54.213 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:54.227 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:00:54.285 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:54.280+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:54.354 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:54.355 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:54.355 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:00:54.355 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:00:54.355 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:54.369 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:00:54.417 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:54.416+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:54.486 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:54.484+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:54.547 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:00:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:00:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:00:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4235.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:38.943514+0000", 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.60413668, 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016454880000000001, 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943514+0000", 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943514+0000", 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943515+0000", 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943527+0000", 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2333e-05 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943569+0000", 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1610000000000003e-05 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943597+0000", 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7773999999999999e-05 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943746+0000", 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000149765 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.943801+0000", 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4623000000000002e-05 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.945052+0000", 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012512599999999999 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:38.945160+0000", 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107336 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4235.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:39.948331+0000", 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.599320225, 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017906630000000001, 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948331+0000", 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948331+0000", 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948332+0000", 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3239999999999999e-06 2024-09-22T01:00:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948356+0000", 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4156e-05 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948386+0000", 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0334000000000001e-05 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948411+0000", 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948650+0000", 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000238951 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.948713+0000", 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2498999999999999e-05 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.950035+0000", 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001321904 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:39.950121+0000", 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6621999999999998e-05 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9 3.1a e25/22 3:5cea7035:::rbd_info:head v 25'2, mlcod=0'0)", 2024-09-22T01:00:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:49.716660+0000", 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8309907110000001, 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00109387, 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716660+0000", 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716660+0000", 2024-09-22T01:00:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716660+0000", 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7399999999999999e-07 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716670+0000", 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4120000000000007e-06 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716697+0000", 2024-09-22T01:00:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7608999999999999e-05 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716709+0000", 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1384e-05 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716792+0000", 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3599999999999999e-05 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.716825+0000", 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2391000000000002e-05 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.717703+0000", 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087801399999999999 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:49.717754+0000", 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1186000000000001e-05 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write 905999~23163] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-22T01:00:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:53.748216+0000", 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.79943514000000004, 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021689729999999998, 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 270 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748216+0000", 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748216+0000", 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748219+0000", 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.864e-06 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748261+0000", 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2574000000000003e-05 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748289+0000", 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7361000000000001e-05 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748302+0000", 2024-09-22T01:00:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3730999999999999e-05 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748408+0000", 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105277 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748628+0000", 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022027400000000001 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-22T01:00:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.748792+0000", 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000164026 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.749901+0000", 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011094779999999999 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750299+0000", 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039761400000000002 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750346+0000", 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7484000000000001e-05 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750385+0000", 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8290000000000001e-05 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4476.0:270 3.19 e27/22 ondisk, result = 0)", 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:53.750193+0000", 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.79745797100000004, 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000197648, 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750193+0000", 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750193+0000", 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750194+0000", 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1060000000000001e-06 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750199+0000", 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0839999999999999e-06 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750213+0000", 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3963e-05 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750221+0000", 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3129999999999999e-06 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750282+0000", 2024-09-22T01:00:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0927000000000001e-05 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750294+0000", 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1922e-05 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.750391+0000", 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6333000000000001e-05 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:278 3.8 e27/22 3:1360246b:::rbd_data.117cf538ee01.0000000000000030:head v 27'4, mlcod=25'2)", 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:53.766812+0000", 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.78083865600000002, 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018967280000000001, 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.766812+0000", 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.766812+0000", 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.766814+0000", 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9e-06 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.766862+0000", 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7976999999999998e-05 2024-09-22T01:00:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.766894+0000", 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2360000000000002e-05 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.766914+0000", 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9791000000000001e-05 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.767070+0000", 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015563299999999999 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.767114+0000", 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3785999999999997e-05 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.768628+0000", 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015139820000000001 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:53.768709+0000", 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1298999999999995e-05 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:54.575 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:54.576 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:00:54.617 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:54.616+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:54.688 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:54.685+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:54.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 3.17 --file /var/log/ceph/exp.3.17.5 2024-09-22T01:00:54.745 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:54.745 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:54.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:00:54.763 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:00:54.800 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:54.797+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:54.890 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:54.885+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:54.945 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:54.945 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:54.945 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:00:54.945 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:00:54.945 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:54.964 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:00:54.991 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:54.985+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:55.004 INFO:teuthology.orchestra.run.smithi002.stdout:111 punch from 0x62fd7ec to 0x6307fb9, (0xa7cd bytes) 2024-09-22T01:00:55.012 INFO:teuthology.orchestra.run.smithi002.stdout:112 punch from 0x255177b to 0x25597b2, (0x8037 bytes) 2024-09-22T01:00:55.012 INFO:teuthology.orchestra.run.smithi002.stdout:114 read 0xe45f93d thru 0xe46a516 (0xabda bytes) 2024-09-22T01:00:55.012 INFO:teuthology.orchestra.run.smithi002.stdout:115 write 0xe1c4b54 thru 0xe1c5756 (0xc03 bytes) 2024-09-22T01:00:55.030 INFO:teuthology.orchestra.run.smithi002.stdout:116 trunc from 0xe58c3ea to 0x711a821 2024-09-22T01:00:55.092 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:55.089+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:55.113 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:00:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4235.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.184392654, 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015255189999999999, 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930032+0000", 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-07 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930049+0000", 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7669999999999999e-05 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930072+0000", 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2232e-05 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930090+0000", 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8491999999999998e-05 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930185+0000", 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4500999999999995e-05 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930236+0000", 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1798000000000001e-05 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.931471+0000", 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001234228 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.931557+0000", 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5853999999999995e-05 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.118 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:55.134 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:00:55.189 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.189 INFO:teuthology.orchestra.run.smithi002.stdout:118 read 0x31c80fc thru 0x31d3ae6 (0xb9eb bytes) 2024-09-22T01:00:55.189 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.189 INFO:teuthology.orchestra.run.smithi002.stdout:122 write 0x622240a thru 0x6228c45 (0x683c bytes) 2024-09-22T01:00:55.192 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:55.188+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:55.257 INFO:teuthology.orchestra.run.smithi002.stdout:123 read 0x6ad750a thru 0x6adbd4a (0x4841 bytes) 2024-09-22T01:00:55.257 INFO:teuthology.orchestra.run.smithi002.stdout:124 read 0x5b80d29 thru 0x5b82bf8 (0x1ed0 bytes) 2024-09-22T01:00:55.257 INFO:teuthology.orchestra.run.smithi002.stdout:125 read 0x10576c8 thru 0x105ccec (0x5625 bytes) 2024-09-22T01:00:55.257 INFO:teuthology.orchestra.run.smithi002.stdout:126 writesame 0x378c434 thru 0x378e302 data_size 0xb(0x1ecf bytes) 2024-09-22T01:00:55.292 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:55.292 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:55.292 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:55.292 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:55.293 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:55.292+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:55.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:00:55.310 INFO:teuthology.orchestra.run.smithi002.stdout:128 write 0x7757bf9 thru 0x7762995 (0xad9d bytes) 2024-09-22T01:00:55.314 INFO:teuthology.orchestra.run.smithi002.stdout:129 punch from 0x4641ed1 to 0x464bf05, (0xa034 bytes) 2024-09-22T01:00:55.314 INFO:teuthology.orchestra.run.smithi002.stdout:130 read 0x37af6a1 thru 0x37bc5cb (0xcf2b bytes) 2024-09-22T01:00:55.316 INFO:teuthology.orchestra.run.smithi002.stdout:131 read 0xa760f3 thru 0xa7afc2 (0x4ed0 bytes) 2024-09-22T01:00:55.339 INFO:teuthology.orchestra.run.smithi002.stdout:132 writesame 0xd147bdf thru 0xd149c29 data_size 0x1(0x204b bytes) 2024-09-22T01:00:55.339 INFO:teuthology.orchestra.run.smithi002.stdout:133 read 0x27da196 thru 0x27e900b (0xee76 bytes) 2024-09-22T01:00:55.350 INFO:teuthology.orchestra.run.smithi002.stdout:135 trunc from 0xd149c2a to 0x38d04e4 2024-09-22T01:00:55.394 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:55.389+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:55.445 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:55.445 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:55.445 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:00:55.445 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:00:55.446 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:55.463 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:00:55.469 INFO:teuthology.orchestra.run.smithi002.stdout:138 punch from 0x6353a to 0x66a13, (0x34d9 bytes) 2024-09-22T01:00:55.469 INFO:teuthology.orchestra.run.smithi002.stdout:139 write 0x3436cfe thru 0x343f4d6 (0x87d9 bytes) 2024-09-22T01:00:55.473 INFO:teuthology.orchestra.run.smithi002.stdout:142 read 0x128a634 thru 0x1292cd1 (0x869e bytes) 2024-09-22T01:00:55.473 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.473 INFO:teuthology.orchestra.run.smithi002.stdout:145 punch from 0x69a05b to 0x6a6eef, (0xce94 bytes) 2024-09-22T01:00:55.486 INFO:teuthology.orchestra.run.smithi002.stdout:146 writesame 0x8cd2acc thru 0x8cd58e4 data_size 0x1(0x2e19 bytes) 2024-09-22T01:00:55.486 INFO:teuthology.orchestra.run.smithi002.stdout:147 write 0x11b3b15 thru 0x11b6b27 (0x3013 bytes) 2024-09-22T01:00:55.494 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:55.492+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:55.529 INFO:teuthology.orchestra.run.smithi002.stdout:148 punch from 0x25f9148 to 0x25fff16, (0x6dce bytes) 2024-09-22T01:00:55.529 INFO:teuthology.orchestra.run.smithi002.stdout:150 punch from 0x5ed210 to 0x5f09a2, (0x3792 bytes) 2024-09-22T01:00:55.529 INFO:teuthology.orchestra.run.smithi002.stdout:151 read 0x284648d thru 0x2848ff2 (0x2b66 bytes) 2024-09-22T01:00:55.536 INFO:teuthology.orchestra.run.smithi002.stdout:152 trunc from 0x8cd58e5 to 0x2cf0c20 2024-09-22T01:00:55.545 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.545 INFO:teuthology.orchestra.run.smithi002.stdout:155 write 0x12522fd thru 0x1259336 (0x703a bytes) 2024-09-22T01:00:55.596 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:55.592+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:55.600 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1182 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53895026099999999, 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054296337, 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1182 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:00:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2399999999999999e-07 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061556+0000", 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3700000000000002e-06 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061563+0000", 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0149999999999998e-06 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061568+0000", 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115682+0000", 2024-09-22T01:00:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054113513000000002 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115767+0000", 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4670000000000001e-05 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115850+0000", 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3264999999999996e-05 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1183 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:00:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53892852499999999, 2024-09-22T01:00:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054497446999999997, 2024-09-22T01:00:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1183 2024-09-22T01:00:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061576+0000", 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.65e-07 2024-09-22T01:00:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061585+0000", 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4960000000000002e-06 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061593+0000", 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5340000000000002e-06 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061603+0000", 2024-09-22T01:00:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0213e-05 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115905+0000", 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054302187000000002 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115985+0000", 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9429000000000001e-05 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116073+0000", 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8222999999999999e-05 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1184 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53889319599999996, 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054590382999999999, 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1184 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061613+0000", 2024-09-22T01:00:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061620+0000", 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.781e-06 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061625+0000", 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8999999999999997e-06 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116113+0000", 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054488414999999998 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116154+0000", 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0552000000000001e-05 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116201+0000", 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1185 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.538871926, 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054816747999999998, 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1185 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.34e-07 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061634+0000", 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.996e-06 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061641+0000", 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.455e-06 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061645+0000", 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6979999999999997e-06 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116274+0000", 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054628659000000003 2024-09-22T01:00:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116363+0000", 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8721000000000007e-05 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116449+0000", 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5785000000000006e-05 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1186 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53885183699999994, 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055030429999999998, 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1186 2024-09-22T01:00:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061662+0000", 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1700000000000003e-06 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061669+0000", 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4109999999999999e-06 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061674+0000", 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999999e-06 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116496+0000", 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054821684000000002 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116585+0000", 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9276000000000004e-05 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116682+0000", 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7386000000000006e-05 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1187 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53882181799999995, 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055229227999999998, 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1187 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:00:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061685+0000", 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.289e-06 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061691+0000", 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7710000000000001e-06 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061702+0000", 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0443e-05 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116727+0000", 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055024977000000003 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116811+0000", 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4524000000000006e-05 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116911+0000", 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9802000000000004e-05 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1282 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53784523399999995, 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054493779999999999, 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1282 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062660+0000", 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6500000000000008e-07 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062664+0000", 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2080000000000002e-06 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062676+0000", 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.216e-05 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062684+0000", 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.576e-06 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116948+0000", 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054264616000000002 2024-09-22T01:00:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117035+0000", 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7331999999999997e-05 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117152+0000", 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011692300000000001 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1283 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53780737199999995, 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054680980999999997, 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1283 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062697+0000", 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3900000000000005e-07 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062700+0000", 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062708+0000", 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8019999999999992e-06 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062713+0000", 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609999999999998e-06 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117187+0000", 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054473426999999998 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117270+0000", 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3353e-05 2024-09-22T01:00:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117377+0000", 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010757899999999999 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1284 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53778163000000001, 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054881757000000003, 2024-09-22T01:00:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1284 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062723+0000", 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062725+0000", 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062732+0000", 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062737+0000", 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2580000000000003e-06 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117413+0000", 2024-09-22T01:00:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054675834 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117498+0000", 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4753000000000007e-05 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117604+0000", 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106363 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1285 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53775956000000003, 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055095997000000001, 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1285 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062745+0000", 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062747+0000", 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0760000000000001e-06 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062754+0000", 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.844e-06 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062759+0000", 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9799999999999998e-06 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117639+0000", 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054880802999999999 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117723+0000", 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3746999999999996e-05 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117840+0000", 2024-09-22T01:00:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011718900000000001 2024-09-22T01:00:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1286 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53772531099999998, 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055282537, 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1286 2024-09-22T01:00:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062782+0000", 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.835e-06 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062790+0000", 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.055e-06 2024-09-22T01:00:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062796+0000", 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117873+0000", 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055077470000000003 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117951+0000", 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118061+0000", 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110027 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1287 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53770008499999999, 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055478658, 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1287 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062806+0000", 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.142e-06 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062813+0000", 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.613e-06 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062818+0000", 2024-09-22T01:00:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:00:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118096+0000", 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055277739999999999 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118180+0000", 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4795000000000004e-05 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118282+0000", 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101974 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1288 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53767609199999999, 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055678514999999998, 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1288 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1199999999999998e-07 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062830+0000", 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.114e-06 2024-09-22T01:00:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062838+0000", 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8599999999999993e-06 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062843+0000", 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118315+0000", 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055471883999999999 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118402+0000", 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6992999999999999e-05 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118506+0000", 2024-09-22T01:00:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104238 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1289 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.537642905, 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055873772000000002, 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1289 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063911+0000", 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010504500000000001 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063923+0000", 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1805e-05 2024-09-22T01:00:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063947+0000", 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3519999999999998e-05 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063964+0000", 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6966e-05 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118539+0000", 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054574947999999998 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118631+0000", 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2238999999999996e-05 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118735+0000", 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103844 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1290 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53650007899999996, 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054954785999999999, 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1290 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064013+0000", 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064020+0000", 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0310000000000001e-06 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064044+0000", 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4340000000000001e-05 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064081+0000", 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6720000000000001e-05 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118769+0000", 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054687604000000001 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118852+0000", 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118959+0000", 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106969 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1291 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53639980499999995, 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055078496999999997, 2024-09-22T01:00:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1291 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064105+0000", 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.268e-06 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064154+0000", 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9122999999999997e-05 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064174+0000", 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0057000000000001e-05 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064189+0000", 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4083000000000001e-05 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118992+0000", 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054803604999999998 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119079+0000", 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7109999999999998e-05 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119183+0000", 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103251 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1292 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.536293503, 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055189561999999998, 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1292 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065373+0000", 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001162419 2024-09-22T01:00:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065382+0000", 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6979999999999993e-06 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065403+0000", 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0676e-05 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065418+0000", 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.468e-05 2024-09-22T01:00:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119218+0000", 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053799877000000003 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119301+0000", 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2910000000000004e-05 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119400+0000", 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9302000000000006e-05 2024-09-22T01:00:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1295 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53493424300000003, 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054395285000000002, 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1295 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065571+0000", 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065575+0000", 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065590+0000", 2024-09-22T01:00:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.456e-05 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065616+0000", 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5935000000000002e-05 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119786+0000", 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054170124 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119862+0000", 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5833000000000004e-05 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119965+0000", 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103281 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1296 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1327104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53487193600000005, 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054554510000000001, 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1296 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066090+0000", 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045860600000000002 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066104+0000", 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3883000000000001e-05 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066125+0000", 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0392000000000001e-05 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066139+0000", 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3925e-05 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120000+0000", 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053860906 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120084+0000", 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4875000000000003e-05 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120186+0000", 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010192300000000001 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1297 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53434287599999997, 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054246075999999997, 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1297 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066728+0000", 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056672199999999997 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066733+0000", 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7400000000000001e-06 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066745+0000", 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.166e-05 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066753+0000", 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1620000000000008e-06 2024-09-22T01:00:55.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120220+0000", 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05346687 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120303+0000", 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3354000000000002e-05 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120407+0000", 2024-09-22T01:00:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103568 2024-09-22T01:00:55.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.696 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:55.696 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:55.696 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:55.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:00:55.697 INFO:teuthology.orchestra.run.smithi002.stdout:156 trunc from 0x2cf0c20 to 0x8a5f082 2024-09-22T01:00:55.697 INFO:teuthology.orchestra.run.smithi002.stdout:157 write 0x7f757b0 thru 0x7f7bc8a (0x64db bytes) 2024-09-22T01:00:55.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:00:55.699 INFO:teuthology.orchestra.run.smithi002.stdout:158 write 0x63cb211 thru 0x63da61c (0xf40c bytes) 2024-09-22T01:00:55.700 INFO:teuthology.orchestra.run.smithi002.stdout:159 read 0x19aebca thru 0x19bd0a1 (0xe4d8 bytes) 2024-09-22T01:00:55.700 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.700 INFO:teuthology.orchestra.run.smithi002.stdout:161 writesame 0x8c9a48a thru 0x8c9dca1 data_size 0x14(0x3818 bytes) 2024-09-22T01:00:55.700 INFO:teuthology.orchestra.run.smithi002.stdout:164 writesame 0x2a8707a thru 0x2a908c9 data_size 0x8(0x9850 bytes) 2024-09-22T01:00:55.700 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:55.696+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:55.727 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.738 INFO:teuthology.orchestra.run.smithi002.stdout:166 trunc from 0x8c9dca2 to 0x176b518 2024-09-22T01:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:00:55.821 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:55.820+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:55.830 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:00:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4476.0:5939 3.11 e29/22 3:8e995c1f:::rbd_object_map.117c1b47bdd2:head v 29'80, mlcod=29'78)", 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.830714+0000", 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0011665079999999999, 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012128340000000001, 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.830714+0000", 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.830714+0000", 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.830715+0000", 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4729999999999999e-06 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.830725+0000", 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.830746+0000", 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0800000000000001e-05 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.830758+0000", 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2293000000000001e-05 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.830880+0000", 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121679 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.830931+0000", 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1294000000000001e-05 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 1 2024-09-22T01:00:55.834 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:55.835 INFO:teuthology.orchestra.run.smithi002.stdout:167 read 0x35eef9 thru 0x369cbd (0xadc5 bytes) 2024-09-22T01:00:55.845 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:00:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:169 trunc from 0x176b518 to 0x75c45e2 2024-09-22T01:00:55.852 INFO:teuthology.orchestra.run.smithi002.stdout:170 writesame 0x47d9441 thru 0x47e2222 data_size 0x1a(0x8de2 bytes) 2024-09-22T01:00:55.868 INFO:teuthology.orchestra.run.smithi002.stdout:171 trunc from 0x75c45e2 to 0xb140a2f 2024-09-22T01:00:55.875 INFO:teuthology.orchestra.run.smithi002.stdout:173 writesame 0x18f1825 thru 0x190020e data_size 0x4f(0xe9ea bytes) 2024-09-22T01:00:55.879 INFO:teuthology.orchestra.run.smithi002.stdout:176 read 0x723c0b9 thru 0x724264a (0x6592 bytes) 2024-09-22T01:00:55.879 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.879 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.890 INFO:teuthology.orchestra.run.smithi002.stdout:179 trunc from 0xb140a2f to 0x3bb3589 2024-09-22T01:00:55.898 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:55.893+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:55.902 INFO:teuthology.orchestra.run.smithi002.stdout:181 read 0x9a2e43 thru 0x9a814b (0x5309 bytes) 2024-09-22T01:00:55.907 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:00:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:00:55.909 INFO:teuthology.orchestra.run.smithi002.stdout:182 trunc from 0x3bb3589 to 0x843ea6c 2024-09-22T01:00:55.915 INFO:teuthology.orchestra.run.smithi002.stdout:183 trunc from 0x843ea6c to 0x885abe2 2024-09-22T01:00:55.921 INFO:teuthology.orchestra.run.smithi002.stdout:186 read 0x353bb33 thru 0x35477cb (0xbc99 bytes) 2024-09-22T01:00:55.921 INFO:teuthology.orchestra.run.smithi002.stdout:188 read 0x3637dd8 thru 0x364112e (0x9357 bytes) 2024-09-22T01:00:55.921 INFO:teuthology.orchestra.run.smithi002.stdout:189 trunc from 0x885abe2 to 0x8b16312 2024-09-22T01:00:55.924 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:00:55.925 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:00:55.927 INFO:teuthology.orchestra.run.smithi002.stdout:190 read 0x83c4d06 thru 0x83cd8c7 (0x8bc2 bytes) 2024-09-22T01:00:55.927 INFO:teuthology.orchestra.run.smithi002.stdout:191 trunc from 0x8b16312 to 0x8670b07 2024-09-22T01:00:55.934 INFO:teuthology.orchestra.run.smithi002.stdout:192 read 0x4d67d85 thru 0x4d6c27a (0x44f6 bytes) 2024-09-22T01:00:55.934 INFO:teuthology.orchestra.run.smithi002.stdout:193 write 0x42b6493 thru 0x42c5e8a (0xf9f8 bytes) 2024-09-22T01:00:55.942 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:00:55.942 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:00:55.950 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:55.950 INFO:teuthology.orchestra.run.smithi002.stdout:195 read 0x3bf2451 thru 0x3bf89f3 (0x65a3 bytes) 2024-09-22T01:00:55.950 INFO:teuthology.orchestra.run.smithi002.stdout:196 write 0x3eee77c thru 0x3ef66d3 (0x7f58 bytes) 2024-09-22T01:00:55.961 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:00:55.961 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:00:55.962 INFO:teuthology.orchestra.run.smithi002.stdout:197 read 0x52fd544 thru 0x530b8e8 (0xe3a5 bytes) 2024-09-22T01:00:55.962 INFO:teuthology.orchestra.run.smithi002.stdout:198 write 0x55a429d thru 0x55a93a8 (0x510c bytes) 2024-09-22T01:00:55.972 INFO:teuthology.orchestra.run.smithi002.stdout:199 trunc from 0x8670b07 to 0xd91b6f0 2024-09-22T01:00:55.980 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:00:55.981 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:00:55.981 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:00:55.983 INFO:teuthology.orchestra.run.smithi002.stdout:200 write 0x33c6cfb thru 0x33c7dca (0x10d0 bytes) 2024-09-22T01:00:55.985 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:55.985 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:55.998 INFO:teuthology.orchestra.run.smithi002.stdout:203 trunc from 0xd91b6f0 to 0x4af8160 2024-09-22T01:00:55.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:00:56.000 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:55.996+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:56.000 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:00:56.000 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:00:56.018 INFO:teuthology.orchestra.run.smithi002.stdout:204 trunc from 0x4af8160 to 0x8da7bf2 2024-09-22T01:00:56.018 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:00:56.018 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:00:56.021 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:00:56.022 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:00:56.027 INFO:teuthology.orchestra.run.smithi002.stdout:205 write 0x6ebd35f thru 0x6ebf5e2 (0x2284 bytes) 2024-09-22T01:00:56.033 INFO:teuthology.orchestra.run.smithi002.stdout:207 punch from 0x670444f to 0x670538b, (0xf3c bytes) 2024-09-22T01:00:56.043 INFO:teuthology.orchestra.run.smithi002.stdout:208 write 0xb60e72b thru 0xb611c2c (0x3502 bytes) 2024-09-22T01:00:56.047 INFO:teuthology.orchestra.run.smithi002.stdout:209 write 0x4d1739 thru 0x4d5d4c (0x4614 bytes) 2024-09-22T01:00:56.049 INFO:teuthology.orchestra.run.smithi002.stdout:210 read 0x8a920df thru 0x8a9878f (0x66b1 bytes) 2024-09-22T01:00:56.049 INFO:teuthology.orchestra.run.smithi002.stdout:212 clone 2 order 18 su 131072 sc 5 2024-09-22T01:00:56.100 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:56.096+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:56.160 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:56.160 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:56.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:00:56.201 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:56.196+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:56.291 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:56.291 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:56.301 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:00:56.300+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:00:56.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:00:56.402 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:56.400+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:56.419 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:56.419 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:56.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:00:56.503 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:56.497+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:56.557 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:56.557 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:56.569 INFO:teuthology.orchestra.run.smithi119.stderr:Exporting 3.17 info 3.17( empty local-lis/les=22/23 n=0 ec=22/22 lis/c=22/22 les/c/f=23/23/0 sis=22) 2024-09-22T01:00:56.570 INFO:teuthology.orchestra.run.smithi119.stderr:Export successful 2024-09-22T01:00:56.574 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:00:56.616 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:56.613+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:56.705 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:56.700+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:56.731 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:56.731 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:56.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:00:56.805 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:56.804+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:56.878 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:56.878 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:56.893 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:00:56.893 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone1 from 0xb611c2d (overlap 0x176b518) to 0x706987 2024-09-22T01:00:56.940 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:56.937+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:57.007 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:57.001+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:57.042 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:57.042 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:57.056 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:00:57.080 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-22T01:00:57.108 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:57.105+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:57.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 3.17 2024-09-22T01:00:57.198 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:57.198 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:57.215 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:00:57.224 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:57.221+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:57.309 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:57.308+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:57.393 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:57.393 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:57.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:00:57.410 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:57.408+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:57.487 INFO:teuthology.orchestra.run.smithi002.stdout:216 write 0x49bce99 thru 0x49c4621 (0x7789 bytes) 2024-09-22T01:00:57.504 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:57.510 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:57.505+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:57.514 INFO:teuthology.orchestra.run.smithi002.stdout:218 write 0xc05c78f thru 0xc0604b2 (0x3d24 bytes) 2024-09-22T01:00:57.519 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:57.519 INFO:teuthology.orchestra.run.smithi002.stdout:221 write 0x5e6755d thru 0x5e6ecfd (0x77a1 bytes) 2024-09-22T01:00:57.523 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:57.524 INFO:teuthology.orchestra.run.smithi002.stdout:224 punch from 0x22d8d6d to 0x22e638a, (0xd61d bytes) 2024-09-22T01:00:57.524 INFO:teuthology.orchestra.run.smithi002.stdout:225 write 0x64e960b thru 0x64efa30 (0x6426 bytes) 2024-09-22T01:00:57.528 INFO:teuthology.orchestra.run.smithi002.stdout:226 writesame 0x9b12698 thru 0x9b1bf67 data_size 0x50(0x98d0 bytes) 2024-09-22T01:00:57.543 INFO:teuthology.orchestra.run.smithi002.stdout:227 write 0xc846de3 thru 0xc8510ec (0xa30a bytes) 2024-09-22T01:00:57.548 INFO:teuthology.orchestra.run.smithi002.stdout:228 punch from 0x265f380 to 0x266033e, (0xfbe bytes) 2024-09-22T01:00:57.553 INFO:teuthology.orchestra.run.smithi002.stdout:229 trunc from 0xc8510ed to 0xa33ae1a 2024-09-22T01:00:57.584 INFO:teuthology.orchestra.run.smithi002.stdout:230 punch from 0xaaafa3 to 0xab5dff, (0xae5c bytes) 2024-09-22T01:00:57.584 INFO:teuthology.orchestra.run.smithi002.stdout:231 read 0x28ebac5 thru 0x28f9f1e (0xe45a bytes) 2024-09-22T01:00:57.585 INFO:teuthology.orchestra.run.smithi002.stdout:232 read 0x92e78f thru 0x937f91 (0x9803 bytes) 2024-09-22T01:00:57.587 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:57.587 INFO:teuthology.orchestra.run.smithi002.stdout:238 write 0x5537298 thru 0x55413d2 (0xa13b bytes) 2024-09-22T01:00:57.588 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:57.588 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:57.607 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:00:57.608 INFO:teuthology.orchestra.run.smithi002.stdout:239 trunc from 0xa33ae1a to 0x48f98a5 2024-09-22T01:00:57.653 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:57.652+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:57.662 INFO:teuthology.orchestra.run.smithi002.stdout:240 writesame 0x84eee27 thru 0x84f0432 data_size 0x4(0x160c bytes) 2024-09-22T01:00:57.684 INFO:teuthology.orchestra.run.smithi002.stdout:242 write 0xd568281 thru 0xd572edb (0xac5b bytes) 2024-09-22T01:00:57.692 INFO:teuthology.orchestra.run.smithi002.stdout:243 read 0x602418f thru 0x602f240 (0xb0b2 bytes) 2024-09-22T01:00:57.692 INFO:teuthology.orchestra.run.smithi002.stdout:244 write 0x32b568e thru 0x32b85f1 (0x2f64 bytes) 2024-09-22T01:00:57.700 INFO:teuthology.orchestra.run.smithi002.stdout:245 write 0x8860c83 thru 0x88621c0 (0x153e bytes) 2024-09-22T01:00:57.706 INFO:teuthology.orchestra.run.smithi002.stdout:246 read 0x9abcc2f thru 0x9ac622b (0x95fd bytes) 2024-09-22T01:00:57.706 INFO:teuthology.orchestra.run.smithi002.stdout:247 read 0x2b31d68 thru 0x2b39a86 (0x7d1f bytes) 2024-09-22T01:00:57.712 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:57.709+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:57.716 INFO:teuthology.orchestra.run.smithi002.stdout:249 write 0x839c1e4 thru 0x83a3112 (0x6f2f bytes) 2024-09-22T01:00:57.722 INFO:teuthology.orchestra.run.smithi002.stdout:252 write 0x4770e81 thru 0x477421f (0x339f bytes) 2024-09-22T01:00:57.729 INFO:teuthology.orchestra.run.smithi002.stdout:253 write 0x901f197 thru 0x902a5f7 (0xb461 bytes) 2024-09-22T01:00:57.738 INFO:teuthology.orchestra.run.smithi002.stdout:254 read 0x5440825 thru 0x544b8b2 (0xb08e bytes) 2024-09-22T01:00:57.738 INFO:teuthology.orchestra.run.smithi002.stdout:255 writesame 0x3aac344 thru 0x3aaed67 data_size 0x6(0x2a24 bytes) 2024-09-22T01:00:57.742 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:57.742 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:57.747 INFO:teuthology.orchestra.run.smithi002.stdout:256 punch from 0x9d9bfb3 to 0x9da98c2, (0xd90f bytes) 2024-09-22T01:00:57.748 INFO:teuthology.orchestra.run.smithi002.stdout:259 write 0xbac91f thru 0xbb36cf (0x6db1 bytes) 2024-09-22T01:00:57.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:00:57.775 INFO:teuthology.orchestra.run.smithi002.stdout:260 trunc from 0xd572edc to 0x9d072d 2024-09-22T01:00:57.779 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:00:57.821 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:57.820+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:57.824 INFO:teuthology.orchestra.run.smithi002.stdout:261 write 0x4319114 thru 0x43215e1 (0x84ce bytes) 2024-09-22T01:00:57.829 INFO:teuthology.orchestra.run.smithi002.stdout:262 punch from 0x39fc834 to 0x3a06118, (0x98e4 bytes) 2024-09-22T01:00:57.829 INFO:teuthology.orchestra.run.smithi002.stdout:263 read 0x18809dc thru 0x1889c26 (0x924b bytes) 2024-09-22T01:00:57.847 INFO:teuthology.orchestra.run.smithi002.stdout:265 write 0xc617476 thru 0xc624043 (0xcbce bytes) 2024-09-22T01:00:57.853 INFO:teuthology.orchestra.run.smithi002.stdout:266 punch from 0x58eafa1 to 0x58f4f73, (0x9fd2 bytes) 2024-09-22T01:00:57.853 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:57.853 INFO:teuthology.orchestra.run.smithi002.stdout:268 writesame 0x3df2707 thru 0x3dfb449 data_size 0x1d(0x8d43 bytes) 2024-09-22T01:00:57.857 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:57.857 INFO:teuthology.orchestra.run.smithi002.stdout:271 write 0x985554 thru 0x98bf9f (0x6a4c bytes) 2024-09-22T01:00:57.872 INFO:teuthology.orchestra.run.smithi002.stdout:272 write 0xdfc2ee1 thru 0xdfcc5be (0x96de bytes) 2024-09-22T01:00:57.876 INFO:teuthology.orchestra.run.smithi002.stdout:274 write 0x2cee775 thru 0x2cfc437 (0xdcc3 bytes) 2024-09-22T01:00:57.880 INFO:teuthology.orchestra.run.smithi002.stdout:275 write 0x9a9a17d thru 0x9a9a2b4 (0x138 bytes) 2024-09-22T01:00:57.883 INFO:teuthology.orchestra.run.smithi002.stdout:278 write 0xc418870 thru 0xc4218c7 (0x9058 bytes) 2024-09-22T01:00:57.890 INFO:teuthology.orchestra.run.smithi002.stdout:280 write 0x1142c30 thru 0x114a14b (0x751c bytes) 2024-09-22T01:00:57.894 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:57.894 INFO:teuthology.orchestra.run.smithi002.stdout:283 read 0xb370d24 thru 0xb373eb1 (0x318e bytes) 2024-09-22T01:00:57.894 INFO:teuthology.orchestra.run.smithi002.stdout:288 punch from 0x82e2b46 to 0x82ef860, (0xcd1a bytes) 2024-09-22T01:00:57.901 INFO:teuthology.orchestra.run.smithi002.stdout:290 write 0xe501929 thru 0xe5090c1 (0x7799 bytes) 2024-09-22T01:00:57.905 INFO:teuthology.orchestra.run.smithi002.stdout:291 read 0xe0951a thru 0xe11ad4 (0x85bb bytes) 2024-09-22T01:00:57.913 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:57.913 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:57.913 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:57.912+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:57.921 INFO:teuthology.orchestra.run.smithi002.stdout:292 trunc from 0xe5090c2 to 0x2e302e4 2024-09-22T01:00:57.935 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:00:57.944 INFO:teuthology.orchestra.run.smithi002.stdout:293 punch from 0x433cb1 to 0x43ee02, (0xb151 bytes) 2024-09-22T01:00:57.971 INFO:teuthology.orchestra.run.smithi002.stdout:295 write 0xdaabdb7 thru 0xdab3da6 (0x7ff0 bytes) 2024-09-22T01:00:57.975 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:57.975 INFO:teuthology.orchestra.run.smithi002.stdout:298 punch from 0xac13e4a to 0xac183bd, (0x4573 bytes) 2024-09-22T01:00:57.975 INFO:teuthology.orchestra.run.smithi002.stdout:299 write 0x9740565 thru 0x97420f6 (0x1b92 bytes) 2024-09-22T01:00:57.981 INFO:teuthology.orchestra.run.smithi002.stdout:300 writesame 0x73506d thru 0x73eef2 data_size 0x2(0x9e86 bytes) 2024-09-22T01:00:58.014 INFO:teuthology.orchestra.run.smithi002.stdout:302 read 0xb965291 thru 0xb971c09 (0xc979 bytes) 2024-09-22T01:00:58.014 INFO:teuthology.orchestra.run.smithi002.stdout:305 write 0x14ef169 thru 0x14fe728 (0xf5c0 bytes) 2024-09-22T01:00:58.014 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:58.009+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:58.019 INFO:teuthology.orchestra.run.smithi002.stdout:306 writesame 0xa9b6dc thru 0xaa9597 data_size 0xa(0xdebc bytes) 2024-09-22T01:00:58.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:00:58.026 INFO:teuthology.orchestra.run.smithi002.stdout:308 write 0x3029fbb thru 0x302c1ae (0x21f4 bytes) 2024-09-22T01:00:58.046 INFO:teuthology.orchestra.run.smithi002.stdout:309 writesame 0xddbcc7a thru 0xddc5594 data_size 0x1(0x891b bytes) 2024-09-22T01:00:58.046 INFO:teuthology.orchestra.run.smithi002.stdout:310 punch from 0x4fda3e9 to 0x4fe534d, (0xaf64 bytes) 2024-09-22T01:00:58.046 INFO:teuthology.orchestra.run.smithi002.stdout:311 writesame 0xa9c73e6 thru 0xa9c77b0 data_size 0x1(0x3cb bytes) 2024-09-22T01:00:58.052 INFO:teuthology.orchestra.run.smithi002.stdout:312 write 0xc81eba thru 0xc91aa8 (0xfbef bytes) 2024-09-22T01:00:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:314 write 0x7f0d8f7 thru 0x7f13c43 (0x634d bytes) 2024-09-22T01:00:58.062 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:58.062 INFO:teuthology.orchestra.run.smithi002.stdout:317 write 0xd0807e thru 0xd11192 (0x9115 bytes) 2024-09-22T01:00:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:318 writesame 0x96fce81 thru 0x9701792 data_size 0x2(0x4912 bytes) 2024-09-22T01:00:58.072 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:00:58.072 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:00:58.088 INFO:teuthology.orchestra.run.smithi002.stdout:319 trunc from 0xddc5595 to 0x53a63c2 2024-09-22T01:00:58.094 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:00:58.107 INFO:teuthology.orchestra.run.smithi002.stdout:320 read 0x275fe52 thru 0x2769a11 (0x9bc0 bytes) 2024-09-22T01:00:58.107 INFO:teuthology.orchestra.run.smithi002.stdout:322 writesame 0x1518c11 thru 0x15228dd data_size 0x1(0x9ccd bytes) 2024-09-22T01:00:58.115 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:58.113+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:58.134 INFO:teuthology.orchestra.run.smithi002.stdout:323 punch from 0x4fb1ffa to 0x4fc1758, (0xf75e bytes) 2024-09-22T01:00:58.134 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:58.134 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:58.134 INFO:teuthology.orchestra.run.smithi002.stdout:331 write 0x2d5fd79 thru 0x2d66493 (0x671b bytes) 2024-09-22T01:00:58.138 INFO:teuthology.orchestra.run.smithi002.stdout:332 read 0x17dd670 thru 0x17ea361 (0xccf2 bytes) 2024-09-22T01:00:58.142 INFO:teuthology.orchestra.run.smithi002.stdout:334 trunc from 0x53a63c2 to 0x285bfb7 2024-09-22T01:00:58.155 INFO:teuthology.orchestra.run.smithi002.stdout:335 read 0xabf910 thru 0xac36f4 (0x3de5 bytes) 2024-09-22T01:00:58.169 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:00:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:00:58.179 INFO:teuthology.orchestra.run.smithi002.stdout:336 writesame 0xd523705 thru 0xd5272ea data_size 0x29(0x3be6 bytes) 2024-09-22T01:00:58.179 INFO:teuthology.orchestra.run.smithi002.stdout:339 clone 3 order 25 su 2097152 sc 8 2024-09-22T01:00:58.187 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:00:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:00:58.203 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:00:58.204 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:00:58.216 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:58.212+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:58.222 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:00:58.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:00:58.228 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:58.228 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:58.228 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:58.228 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:58.241 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:00:58.242 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:00:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:00:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:00:58.263 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:00:58.264 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:00:58.284 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:00:58.284 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:00:58.287 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:00:58.288 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:00:58.333 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:58.332+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:58.354 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:58.354 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:58.354 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:00:58.354 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:00:58.354 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:58.367 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:00:58.418 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:58.413+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:58.519 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:58.517+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4235.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.607619473, 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015255189999999999, 2024-09-22T01:00:58.540 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930032+0000", 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-07 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930049+0000", 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7669999999999999e-05 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930072+0000", 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2232e-05 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930090+0000", 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8491999999999998e-05 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930185+0000", 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4500999999999995e-05 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930236+0000", 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1798000000000001e-05 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.931471+0000", 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001234228 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.931557+0000", 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5853999999999995e-05 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:1939 3.0 e28/22 3:000ab5cf:::rbd_data.117c1b47bdd2.0000000000000039:head v 28'5, mlcod=25'3)", 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.127716+0000", 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4099346000000001, 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010060609999999999, 2024-09-22T01:00:58.545 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.127716+0000", 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.127716+0000", 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.127716+0000", 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.127723+0000", 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6640000000000001e-06 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.127741+0000", 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8351000000000001e-05 2024-09-22T01:00:58.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.127752+0000", 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1066e-05 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.127893+0000", 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014076 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.127917+0000", 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4091e-05 2024-09-22T01:00:58.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.128667+0000", 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00074948799999999998 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.128722+0000", 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5290000000000001e-05 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:1943 3.0 e28/22 3:046f320d:::rbd_data.117c1b47bdd2.000000000000003b:head v 28'6, mlcod=25'3)", 2024-09-22T01:00:58.549 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4075334860000002, 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017318469999999999, 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130118+0000", 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3900000000000004e-07 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130185+0000", 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7076999999999995e-05 2024-09-22T01:00:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130201+0000", 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6320999999999998e-05 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130210+0000", 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9279999999999999e-06 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130349+0000", 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013925799999999999 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130372+0000", 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2660999999999999e-05 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.131801+0000", 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001429034 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.131849+0000", 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7729000000000003e-05 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:1951 3.4 e28/22 3:2544f864:::rbd_data.117c1b47bdd2.000000000000003f:head v 28'3, mlcod=25'1)", 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.159797+0000", 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.3778534599999999, 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069787199999999999, 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.159797+0000", 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.159797+0000", 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.159798+0000", 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.31e-06 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.159809+0000", 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0987e-05 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.159829+0000", 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9199e-05 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.159838+0000", 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1889999999999996e-06 2024-09-22T01:00:58.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.159953+0000", 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115763 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.159975+0000", 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1229999999999998e-05 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.160457+0000", 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00048242499999999998 2024-09-22T01:00:58.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.160495+0000", 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7768999999999997e-05 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:1953 3.0 e28/22 3:03a40355:::rbd_data.117c1b47bdd2.0000000000000040:head v 28'7, mlcod=28'5)", 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.169286+0000", 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.3683644199999998, 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060105900000000003, 2024-09-22T01:00:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169286+0000", 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169286+0000", 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169287+0000", 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169294+0000", 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4139999999999997e-06 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169310+0000", 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5648e-05 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169318+0000", 2024-09-22T01:00:58.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4800000000000004e-06 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169379+0000", 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1014999999999998e-05 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169403+0000", 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4816000000000001e-05 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169852+0000", 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044871800000000002 2024-09-22T01:00:58.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.169887+0000", 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5064000000000002e-05 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:2363 3.0 e28/22 3:000ab5cf:::rbd_data.117c1b47bdd2.0000000000000039:head v 28'8, mlcod=28'6)", 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.312588+0000", 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.2250628959999998, 2024-09-22T01:00:58.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012734720000000001, 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312588+0000", 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312588+0000", 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312589+0000", 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4360000000000001e-06 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312602+0000", 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3334e-05 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312625+0000", 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2308000000000001e-05 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312634+0000", 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.329e-06 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312699+0000", 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5129999999999995e-05 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312725+0000", 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6026000000000001e-05 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313812+0000", 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001086547 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313861+0000", 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9361999999999999e-05 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.566 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:2365 3.4 e28/22 3:2544f864:::rbd_data.117c1b47bdd2.000000000000003f:head v 28'4, mlcod=25'2)", 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.312661+0000", 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.2249893959999998, 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001389995, 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312661+0000", 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312661+0000", 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312662+0000", 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4000000000000002e-07 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312671+0000", 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9210000000000004e-06 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312679+0000", 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9689999999999994e-06 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312685+0000", 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.147e-06 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312873+0000", 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018890899999999999 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.312893+0000", 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9055e-05 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.314010+0000", 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011173240000000001 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.314051+0000", 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1230000000000003e-05 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:2366 3.0 e28/22 3:03a40355:::rbd_data.117c1b47bdd2.0000000000000040:head v 28'9, mlcod=28'6)", 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.313283+0000", 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.2243673419999999, 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011708269999999999, 2024-09-22T01:00:58.571 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313283+0000", 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313283+0000", 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313284+0000", 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9299999999999997e-07 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313294+0000", 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8980000000000008e-06 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313308+0000", 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4279e-05 2024-09-22T01:00:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313315+0000", 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.418e-06 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313384+0000", 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8881999999999995e-05 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.313406+0000", 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1925000000000001e-05 2024-09-22T01:00:58.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.314413+0000", 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001006563 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.314454+0000", 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1168999999999998e-05 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:2395 3.0 e28/22 3:000ab5cf:::rbd_data.117c1b47bdd2.0000000000000039:head v 28'10, mlcod=28'8)", 2024-09-22T01:00:58.575 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.356771+0000", 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.1808795299999999, 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012701920000000001, 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.356771+0000", 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.356771+0000", 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.356772+0000", 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.426e-06 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.356785+0000", 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2446000000000001e-05 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.356805+0000", 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9729e-05 2024-09-22T01:00:58.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.356814+0000", 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8779999999999993e-06 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.356910+0000", 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5385999999999998e-05 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.356932+0000", 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2323999999999998e-05 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357999+0000", 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001066535 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358041+0000", 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2468000000000001e-05 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:2397 3.0 e28/22 3:046f320d:::rbd_data.117c1b47bdd2.000000000000003b:head v 28'11, mlcod=28'8)", 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.357644+0000", 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.1800060569999999, 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011580519999999999, 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357644+0000", 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357644+0000", 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357645+0000", 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2399999999999997e-07 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357652+0000", 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2700000000000001e-06 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357665+0000", 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3078000000000001e-05 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357673+0000", 2024-09-22T01:00:58.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3969999999999994e-06 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357734+0000", 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0609000000000002e-05 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.357755+0000", 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1348999999999998e-05 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358755+0000", 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010004110000000001 2024-09-22T01:00:58.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358802+0000", 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7114000000000003e-05 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:2401 3.4 e28/22 3:2544f864:::rbd_data.117c1b47bdd2.000000000000003f:head v 28'5, mlcod=28'3)", 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.358554+0000", 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.179096913, 2024-09-22T01:00:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092988000000000001, 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358554+0000", 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358554+0000", 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358554+0000", 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0299999999999998e-07 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358559+0000", 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9110000000000001e-06 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358577+0000", 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8050000000000002e-05 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358585+0000", 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7970000000000001e-06 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358658+0000", 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2738999999999996e-05 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.358672+0000", 2024-09-22T01:00:58.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.381e-05 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359429+0000", 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075722099999999998 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359483+0000", 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4648999999999998e-05 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:2402 3.0 e28/22 3:03a40355:::rbd_data.117c1b47bdd2.0000000000000040:head v 28'12, mlcod=28'10)", 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.359253+0000", 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.1783973099999998, 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065118900000000004, 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359253+0000", 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359253+0000", 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359254+0000", 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8599999999999998e-07 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359258+0000", 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5950000000000001e-06 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359268+0000", 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7089999999999997e-06 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359276+0000", 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9160000000000001e-06 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359327+0000", 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1044000000000001e-05 2024-09-22T01:00:58.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359350+0000", 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2395000000000002e-05 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359855+0000", 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050504800000000002 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.359904+0000", 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9796000000000002e-05 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.592 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:4310 3.0 e28/22 3:000ab5cf:::rbd_data.117c1b47bdd2.0000000000000039:head v 28'13, mlcod=28'11)", 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.622075+0000", 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.9155759059999999, 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065557400000000002, 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622075+0000", 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622075+0000", 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622076+0000", 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3799999999999999e-06 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622089+0000", 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3329999999999999e-05 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622106+0000", 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6994e-05 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622113+0000", 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8870000000000003e-06 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622161+0000", 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7669e-05 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622176+0000", 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5031999999999999e-05 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622701+0000", 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00052556399999999996 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.622730+0000", 2024-09-22T01:00:58.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8717999999999998e-05 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:4312 3.0 e28/22 3:046f320d:::rbd_data.117c1b47bdd2.000000000000003b:head v 28'14, mlcod=28'12)", 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.624286+0000", 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.9133646280000001, 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056569499999999996, 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624286+0000", 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624286+0000", 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624286+0000", 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5300000000000004e-07 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624302+0000", 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5361000000000001e-05 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624313+0000", 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0933e-05 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624318+0000", 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624355+0000", 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6987999999999999e-05 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624369+0000", 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4776e-05 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624824+0000", 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045508899999999999 2024-09-22T01:00:58.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.624852+0000", 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7061999999999999e-05 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:5185 3.4 e28/22 3:2378af9a:::rbd_data.117c1b47bdd2.000000000000000d:head v 28'6, mlcod=28'4)", 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.717285+0000", 2024-09-22T01:00:58.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.8203658320000002, 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087931000000000001, 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.717285+0000", 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.717285+0000", 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.717285+0000", 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.717297+0000", 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2156e-05 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.717309+0000", 2024-09-22T01:00:58.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1725e-05 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.717318+0000", 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7530000000000007e-06 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.717375+0000", 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6773e-05 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.717395+0000", 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9925e-05 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.718120+0000", 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072577700000000004 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.718164+0000", 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3523999999999999e-05 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:5214 3.0 e28/22 3:000ab5cf:::rbd_data.117c1b47bdd2.0000000000000039:head v 28'15, mlcod=28'13)", 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.746165+0000", 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.7914853380000002, 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012069719999999999, 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746165+0000", 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746165+0000", 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746167+0000", 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9709999999999998e-06 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746179+0000", 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.169e-05 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746201+0000", 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1733999999999999e-05 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746213+0000", 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2408999999999999e-05 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746285+0000", 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2373000000000007e-05 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746317+0000", 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1231e-05 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.747328+0000", 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001011877 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.747372+0000", 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3686999999999997e-05 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:5216 3.0 e28/22 3:046f320d:::rbd_data.117c1b47bdd2.000000000000003b:head v 28'16, mlcod=28'13)", 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.746613+0000", 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.7910373339999999, 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00098825600000000007, 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746613+0000", 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746613+0000", 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746614+0000", 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1299999999999999e-07 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746621+0000", 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2960000000000001e-06 2024-09-22T01:00:58.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746638+0000", 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6426000000000002e-05 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746645+0000", 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5619999999999998e-06 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746706+0000", 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0341999999999999e-05 2024-09-22T01:00:58.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.746726+0000", 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0207000000000002e-05 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.747563+0000", 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083704099999999996 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.747601+0000", 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8569000000000003e-05 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:5220 3.4 e29/22 3:2378af9a:::rbd_data.117c1b47bdd2.000000000000000d:head v 29'7, mlcod=28'5)", 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.755439+0000", 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.7822116019999998, 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00102648, 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.614 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.755439+0000", 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.755439+0000", 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.755440+0000", 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9400000000000005e-07 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.755448+0000", 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.816e-06 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.755464+0000", 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5888000000000001e-05 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.755475+0000", 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0467e-05 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.755569+0000", 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3821999999999997e-05 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.755593+0000", 2024-09-22T01:00:58.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4805000000000001e-05 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.756410+0000", 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081695200000000002 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.756465+0000", 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5036e-05 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:5981 3.4 e29/22 3:2378af9a:::rbd_data.117c1b47bdd2.000000000000000d:head v 29'8, mlcod=28'6)", 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.878315+0000", 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.659335096, 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012989869999999999, 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.878315+0000", 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.878315+0000", 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.878317+0000", 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.657e-06 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.878328+0000", 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-22T01:00:58.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.878349+0000", 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.154e-05 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.878360+0000", 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0597999999999999e-05 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:58.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.878515+0000", 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000154924 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.878537+0000", 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.287e-05 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.879563+0000", 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010260169999999999 2024-09-22T01:00:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.879614+0000", 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0890999999999998e-05 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:58.623 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:58.625 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:00:58.625 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:58.620+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:58.725 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:58.721+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:58.763 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:00:58.803 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:58.803 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:58.803 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:00:58.804 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:58.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:00:58.826 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:58.821+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:58.907 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone2 from 0xd5272eb (overlap 0x9d072d) to 0x5e3781 2024-09-22T01:00:58.927 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:58.924+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:58.968 INFO:teuthology.orchestra.run.smithi119.stdout: marking collection for removal 2024-09-22T01:00:58.968 INFO:teuthology.orchestra.run.smithi119.stdout:setting '_remove' omap key 2024-09-22T01:00:58.968 INFO:teuthology.orchestra.run.smithi119.stdout:finish_remove_pgs 3.17_head removing 3.17 2024-09-22T01:00:58.970 INFO:teuthology.orchestra.run.smithi119.stdout:Remove successful 2024-09-22T01:00:58.999 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:00:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:00:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:00:59.000 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:59.016 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:00:59.040 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-22T01:00:59.040 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:00:59.036+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:00:59.057 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:59.058 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:59.069 INFO:teuthology.orchestra.run.smithi002.stdout:342 trunc from 0xd5272eb to 0x57c8f2a 2024-09-22T01:00:59.128 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:59.124+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:59.175 INFO:teuthology.orchestra.run.smithi002.stdout:343 writesame 0xcb00d37 thru 0xcb108d7 data_size 0x1(0xfba1 bytes) 2024-09-22T01:00:59.177 INFO:teuthology.orchestra.run.smithi002.stdout:344 read 0x769e028 thru 0x76aa411 (0xc3ea bytes) 2024-09-22T01:00:59.178 INFO:teuthology.orchestra.run.smithi002.stdout:345 read 0x23b270d thru 0x23b9a67 (0x735b bytes) 2024-09-22T01:00:59.178 INFO:teuthology.orchestra.run.smithi002.stdout:346 writesame 0x8990454 thru 0x899a79c data_size 0x1(0xa349 bytes) 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1182 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1375805669999997, 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054296337, 2024-09-22T01:00:59.199 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1182 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2399999999999999e-07 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061556+0000", 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3700000000000002e-06 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061563+0000", 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0149999999999998e-06 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061568+0000", 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115682+0000", 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054113513000000002 2024-09-22T01:00:59.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115767+0000", 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4670000000000001e-05 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115850+0000", 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3264999999999996e-05 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1183 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1375588309999998, 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054497446999999997, 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1183 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061576+0000", 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.65e-07 2024-09-22T01:00:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061585+0000", 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4960000000000002e-06 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061593+0000", 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5340000000000002e-06 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061603+0000", 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0213e-05 2024-09-22T01:00:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115905+0000", 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054302187000000002 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115985+0000", 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9429000000000001e-05 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116073+0000", 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8222999999999999e-05 2024-09-22T01:00:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1184 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1375235019999996, 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054590382999999999, 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1184 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061613+0000", 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061620+0000", 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.781e-06 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061625+0000", 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8999999999999997e-06 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116113+0000", 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054488414999999998 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116154+0000", 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0552000000000001e-05 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116201+0000", 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1185 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1375022320000001, 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054816747999999998, 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1185 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.34e-07 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061634+0000", 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.996e-06 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061641+0000", 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.455e-06 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061645+0000", 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6979999999999997e-06 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116274+0000", 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054628659000000003 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116363+0000", 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8721000000000007e-05 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116449+0000", 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5785000000000006e-05 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1186 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:00:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1374821429999997, 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055030429999999998, 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1186 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061662+0000", 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1700000000000003e-06 2024-09-22T01:00:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061669+0000", 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4109999999999999e-06 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061674+0000", 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999999e-06 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116496+0000", 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054821684000000002 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116585+0000", 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9276000000000004e-05 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116682+0000", 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7386000000000006e-05 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1187 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1374521240000002, 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055229227999999998, 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1187 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061685+0000", 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.289e-06 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061691+0000", 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7710000000000001e-06 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061702+0000", 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0443e-05 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116727+0000", 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055024977000000003 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116811+0000", 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4524000000000006e-05 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116911+0000", 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9802000000000004e-05 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1282 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1364755400000002, 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054493779999999999, 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1282 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062660+0000", 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6500000000000008e-07 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062664+0000", 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2080000000000002e-06 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062676+0000", 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.216e-05 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062684+0000", 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.576e-06 2024-09-22T01:00:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116948+0000", 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054264616000000002 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117035+0000", 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7331999999999997e-05 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117152+0000", 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011692300000000001 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1283 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1364376780000001, 2024-09-22T01:00:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054680980999999997, 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1283 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062697+0000", 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3900000000000005e-07 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062700+0000", 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062708+0000", 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8019999999999992e-06 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062713+0000", 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609999999999998e-06 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117187+0000", 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054473426999999998 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117270+0000", 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3353e-05 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117377+0000", 2024-09-22T01:00:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010757899999999999 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1284 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.136411936, 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054881757000000003, 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1284 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062723+0000", 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062725+0000", 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062732+0000", 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062737+0000", 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2580000000000003e-06 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117413+0000", 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054675834 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117498+0000", 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4753000000000007e-05 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117604+0000", 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106363 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1285 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.136389866, 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055095997000000001, 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1285 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062745+0000", 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062747+0000", 2024-09-22T01:00:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0760000000000001e-06 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062754+0000", 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.844e-06 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062759+0000", 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9799999999999998e-06 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117639+0000", 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054880802999999999 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117723+0000", 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3746999999999996e-05 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117840+0000", 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011718900000000001 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1286 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1363556170000004, 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055282537, 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1286 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:00:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062782+0000", 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.835e-06 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062790+0000", 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.055e-06 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062796+0000", 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117873+0000", 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055077470000000003 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117951+0000", 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118061+0000", 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110027 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1287 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1363303909999996, 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055478658, 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1287 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:00:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062806+0000", 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.142e-06 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062813+0000", 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.613e-06 2024-09-22T01:00:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062818+0000", 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118096+0000", 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055277739999999999 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118180+0000", 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4795000000000004e-05 2024-09-22T01:00:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118282+0000", 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101974 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1288 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1363063980000003, 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055678514999999998, 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1288 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:00:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1199999999999998e-07 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062830+0000", 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.114e-06 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062838+0000", 2024-09-22T01:00:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8599999999999993e-06 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062843+0000", 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118315+0000", 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055471883999999999 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118402+0000", 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6992999999999999e-05 2024-09-22T01:00:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118506+0000", 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104238 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1289 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1362732109999998, 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055873772000000002, 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1289 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:00:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063911+0000", 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010504500000000001 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063923+0000", 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1805e-05 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063947+0000", 2024-09-22T01:00:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3519999999999998e-05 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063964+0000", 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6966e-05 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118539+0000", 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054574947999999998 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118631+0000", 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2238999999999996e-05 2024-09-22T01:00:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118735+0000", 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103844 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1290 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1351303850000001, 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054954785999999999, 2024-09-22T01:00:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1290 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064013+0000", 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064020+0000", 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0310000000000001e-06 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064044+0000", 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4340000000000001e-05 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064081+0000", 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6720000000000001e-05 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118769+0000", 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054687604000000001 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118852+0000", 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118959+0000", 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106969 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1291 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1350301109999998, 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055078496999999997, 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1291 2024-09-22T01:00:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064105+0000", 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.268e-06 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064154+0000", 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9122999999999997e-05 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064174+0000", 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0057000000000001e-05 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064189+0000", 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4083000000000001e-05 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118992+0000", 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054803604999999998 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119079+0000", 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7109999999999998e-05 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119183+0000", 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103251 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1292 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.134923809, 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055189561999999998, 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1292 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065373+0000", 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001162419 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065382+0000", 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6979999999999993e-06 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065403+0000", 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0676e-05 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065418+0000", 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.468e-05 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119218+0000", 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053799877000000003 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119301+0000", 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2910000000000004e-05 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119400+0000", 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9302000000000006e-05 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1295 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1335645489999999, 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054395285000000002, 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1295 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065571+0000", 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065575+0000", 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065590+0000", 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.456e-05 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065616+0000", 2024-09-22T01:00:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5935000000000002e-05 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119786+0000", 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054170124 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119862+0000", 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5833000000000004e-05 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119965+0000", 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103281 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1296 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1327104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1335022419999996, 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054554510000000001, 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1296 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066090+0000", 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045860600000000002 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066104+0000", 2024-09-22T01:00:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3883000000000001e-05 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066125+0000", 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0392000000000001e-05 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066139+0000", 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3925e-05 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120000+0000", 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053860906 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120084+0000", 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4875000000000003e-05 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120186+0000", 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010192300000000001 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1297 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1329731819999997, 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054246075999999997, 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1297 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066728+0000", 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056672199999999997 2024-09-22T01:00:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066733+0000", 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7400000000000001e-06 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066745+0000", 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.166e-05 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066753+0000", 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1620000000000008e-06 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120220+0000", 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05346687 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120303+0000", 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3354000000000002e-05 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:00:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120407+0000", 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103568 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:00:59.287 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:00:59.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:348 writesame 0x6486cfc thru 0x648ff37 data_size 0x62(0x923c bytes) 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:349 read 0x6572946 thru 0x657e17d (0xb838 bytes) 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:351 trunc from 0xcb108d8 to 0xd702972 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:352 write 0xdcdba27 thru 0xdce7e4f (0xc429 bytes) 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:353 read 0x6aea53a thru 0x6af1e3d (0x7904 bytes) 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:354 punch from 0xc15ec18 to 0xc161709, (0x2af1 bytes) 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:356 write 0xabe8109 thru 0xabf0f22 (0x8e1a bytes) 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:358 write 0x2434cce thru 0x243dea3 (0x91d6 bytes) 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:359 read 0x3fefb9b thru 0x3ffda33 (0xde99 bytes) 2024-09-22T01:00:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:360 trunc from 0xdce7e50 to 0xd5853d0 2024-09-22T01:00:59.291 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:59.285+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:59.302 INFO:teuthology.orchestra.run.smithi002.stdout:361 trunc from 0xd5853d0 to 0x7a2a310 2024-09-22T01:00:59.316 INFO:teuthology.orchestra.run.smithi002.stdout:363 read 0x3ac1b0f thru 0x3ad0b8a (0xf07c bytes) 2024-09-22T01:00:59.317 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:366 write 0x54e9a10 thru 0x54f1feb (0x85dc bytes) 2024-09-22T01:00:59.321 INFO:teuthology.orchestra.run.smithi002.stdout:368 trunc from 0x7a2a310 to 0x77b5151 2024-09-22T01:00:59.334 INFO:teuthology.orchestra.run.smithi002.stdout:369 punch from 0x170c9e8 to 0x170e413, (0x1a2b bytes) 2024-09-22T01:00:59.334 INFO:teuthology.orchestra.run.smithi002.stdout:370 writesame 0x2406e33 thru 0x240a5a3 data_size 0x9(0x3771 bytes) 2024-09-22T01:00:59.339 INFO:teuthology.orchestra.run.smithi002.stdout:371 write 0x9a95c thru 0xa01bc (0x5861 bytes) 2024-09-22T01:00:59.361 INFO:teuthology.orchestra.run.smithi002.stdout:372 write 0x9e7fac6 thru 0x9e8a765 (0xaca0 bytes) 2024-09-22T01:00:59.363 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:00:59.363 INFO:teuthology.orchestra.run.smithi002.stdout:374 read 0x2922eb0 thru 0x292f0f0 (0xc241 bytes) 2024-09-22T01:00:59.365 INFO:teuthology.orchestra.run.smithi002.stdout:375 read 0x79c4176 thru 0x79d1c83 (0xdb0e bytes) 2024-09-22T01:00:59.389 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:00:59.385+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:00:59.392 INFO:teuthology.orchestra.run.smithi002.stdout:376 trunc from 0x9e8a766 to 0x4336de 2024-09-22T01:00:59.426 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:59.426 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:00:59.426 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:00:59.426 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:59.428 INFO:teuthology.orchestra.run.smithi002.stdout:377 punch from 0x199232 to 0x19dc05, (0x49d3 bytes) 2024-09-22T01:00:59.428 INFO:teuthology.orchestra.run.smithi002.stdout:378 read 0x340ec1 thru 0x34a1ed (0x932d bytes) 2024-09-22T01:00:59.440 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:00:59.455 INFO:teuthology.orchestra.run.smithi002.stdout:379 writesame 0x42f551f thru 0x42fde87 data_size 0x1d(0x8969 bytes) 2024-09-22T01:00:59.457 INFO:teuthology.orchestra.run.smithi002.stdout:380 clone 4 order 23 su 524288 sc 14 2024-09-22T01:00:59.489 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:00:59.485+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:00:59.517 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.17.5 2024-09-22T01:00:59.591 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:00:59.588+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:00:59.608 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:00:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:00:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:00:59.608 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:59.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:00:59.691 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:59.685+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:59.739 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:00:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:00:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:00:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:00:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2131 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2416640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198060+0000", 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5395943049999996, 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047864156999999997, 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2131 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198060+0000", 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198060+0000", 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198061+0000", 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.23e-07 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198064+0000", 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198073+0000", 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4530000000000008e-06 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198080+0000", 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8530000000000003e-06 2024-09-22T01:00:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.245415+0000", 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047334721000000003 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.245604+0000", 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018855 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.245924+0000", 2024-09-22T01:00:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00032068299999999999 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2132 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198089+0000", 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.539565048, 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048233273, 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2132 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198089+0000", 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198089+0000", 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198090+0000", 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4e-07 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198093+0000", 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7659999999999999e-06 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198101+0000", 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4700000000000002e-06 2024-09-22T01:00:59.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198108+0000", 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6100000000000002e-06 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246031+0000", 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047922861999999997 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246137+0000", 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105715 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246323+0000", 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018600999999999999 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2133 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.748 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198117+0000", 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5395377779999997, 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048784292999999999, 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2133 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198117+0000", 2024-09-22T01:00:59.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198117+0000", 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198117+0000", 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1600000000000001e-07 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198120+0000", 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6740000000000001e-06 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198128+0000", 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198153+0000", 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4583000000000002e-05 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246412+0000", 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048258425000000001 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246595+0000", 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018392899999999999 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246901+0000", 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030544300000000002 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2134 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198163+0000", 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5394918259999999, 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049133563999999998, 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2134 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.753 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198163+0000", 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198163+0000", 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198163+0000", 2024-09-22T01:00:59.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198166+0000", 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8729999999999999e-06 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198179+0000", 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2466999999999999e-05 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198186+0000", 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9399999999999996e-06 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247008+0000", 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048822881999999998 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247116+0000", 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107316 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247296+0000", 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000180366 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2135 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2777088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198194+0000", 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.539459956, 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049682965000000003, 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.757 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2135 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198194+0000", 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198194+0000", 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198195+0000", 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198198+0000", 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8499999999999998e-06 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198207+0000", 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6260000000000001e-06 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198215+0000", 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1799999999999996e-06 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247384+0000", 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049169599000000001 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247571+0000", 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000187164 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247877+0000", 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030589899999999999 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2136 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.761 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198227+0000", 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5394275689999999, 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049974031000000002, 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2136 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198227+0000", 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198227+0000", 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198227+0000", 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:00:59.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198230+0000", 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6570000000000001e-06 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198239+0000", 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.653e-06 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198245+0000", 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6800000000000004e-06 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247943+0000", 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049697386000000003 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248033+0000", 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0062000000000006e-05 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248201+0000", 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016796700000000001 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2137 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2957312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198254+0000", 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.539400229, 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050520378999999997, 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.766 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2137 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198254+0000", 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198254+0000", 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198255+0000", 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198257+0000", 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6299999999999998e-06 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198278+0000", 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0919000000000001e-05 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198288+0000", 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.217e-06 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248284+0000", 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049996051 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248470+0000", 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018665399999999999 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248775+0000", 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000304285 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2138 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3047424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198297+0000", 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5393572039999999, 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050790226000000001, 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2138 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198297+0000", 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198297+0000", 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198298+0000", 2024-09-22T01:00:59.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198301+0000", 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1839999999999999e-06 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198310+0000", 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3060000000000003e-06 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198317+0000", 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7499999999999997e-06 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248837+0000", 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050520355000000003 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248925+0000", 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7391000000000004e-05 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.249087+0000", 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016254999999999999 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2139 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3137536~5961] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198326+0000", 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5393283000000002, 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051635923, 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2139 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198326+0000", 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198326+0000", 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198327+0000", 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198344+0000", 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7266e-05 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198353+0000", 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4800000000000007e-06 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198360+0000", 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8600000000000004e-06 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.249167+0000", 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050806401000000001 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.249350+0000", 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000182879 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.249962+0000", 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061242700000000002 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2140 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3227648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198370+0000", 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5392847270000001, 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051957135000000002, 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2140 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198370+0000", 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198370+0000", 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198370+0000", 2024-09-22T01:00:59.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198373+0000", 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0230000000000001e-06 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198382+0000", 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3240000000000003e-06 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198388+0000", 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.849e-06 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250065+0000", 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051676931000000002 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250173+0000", 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107509 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250327+0000", 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015385900000000001 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2141 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198408+0000", 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.539246468, 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052463599, 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2141 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198408+0000", 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198408+0000", 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198409+0000", 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.540000000000001e-07 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198419+0000", 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9960000000000005e-06 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198428+0000", 2024-09-22T01:00:59.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198435+0000", 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250415+0000", 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051980356999999998 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250607+0000", 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019222099999999999 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250871+0000", 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026423799999999997 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2142 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198447+0000", 2024-09-22T01:00:59.788 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5392071009999997, 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052716038, 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2142 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198447+0000", 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198447+0000", 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198448+0000", 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5000000000000001e-07 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198451+0000", 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198460+0000", 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6440000000000006e-06 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198477+0000", 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250934+0000", 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052457384000000003 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251026+0000", 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1852000000000001e-05 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251163+0000", 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013750200000000001 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2143 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3497984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198488+0000", 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5391664509999998, 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053219743, 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.793 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2143 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198488+0000", 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198488+0000", 2024-09-22T01:00:59.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198489+0000", 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198492+0000", 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198502+0000", 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0530000000000001e-05 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198509+0000", 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6630000000000004e-06 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251246+0000", 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052737447999999999 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251436+0000", 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018967300000000001 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251708+0000", 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027182600000000002 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2144 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3588096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198518+0000", 2024-09-22T01:00:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.539136471, 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053478571000000003, 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2144 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198518+0000", 2024-09-22T01:00:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198518+0000", 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198519+0000", 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198521+0000", 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8159999999999998e-06 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198530+0000", 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2600000000000005e-06 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198536+0000", 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8000000000000001e-06 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251768+0000", 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053231906000000002 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251860+0000", 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1848000000000006e-05 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251996+0000", 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000136298 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2145 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198545+0000", 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5391092620000002, 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053976072999999999, 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2145 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.802 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198545+0000", 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198545+0000", 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198546+0000", 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:00:59.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198558+0000", 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2133000000000001e-05 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198567+0000", 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198574+0000", 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0829999999999998e-06 2024-09-22T01:00:59.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252081+0000", 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053506747 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252263+0000", 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018252200000000001 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252521+0000", 2024-09-22T01:00:59.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025806800000000002 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2146 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198612+0000", 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5390423589999997, 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054205883000000003, 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.806 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2146 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198612+0000", 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198612+0000", 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198613+0000", 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.113e-06 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198619+0000", 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.677e-06 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198647+0000", 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8147e-05 2024-09-22T01:00:59.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198657+0000", 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9289999999999993e-06 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252582+0000", 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053924723000000001 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252679+0000", 2024-09-22T01:00:59.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7250999999999994e-05 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252818+0000", 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013904299999999999 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2147 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198669+0000", 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5389857830000002, 2024-09-22T01:00:59.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054677598000000001, 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2147 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198669+0000", 2024-09-22T01:00:59.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198669+0000", 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198669+0000", 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0699999999999996e-07 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198673+0000", 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4829999999999999e-06 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198682+0000", 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4329999999999993e-06 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198689+0000", 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8970000000000002e-06 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252900+0000", 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054210513000000002 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253085+0000", 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018475100000000001 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253346+0000", 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026161400000000002 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2148 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198700+0000", 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5389546269999999, 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054943604, 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2148 2024-09-22T01:00:59.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198700+0000", 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198700+0000", 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198700+0000", 2024-09-22T01:00:59.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198703+0000", 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.813e-06 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198712+0000", 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4300000000000006e-06 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198718+0000", 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7399999999999998e-06 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253407+0000", 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054688771999999997 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253495+0000", 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8108000000000004e-05 2024-09-22T01:00:59.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253643+0000", 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000148114 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2149 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 4038656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198727+0000", 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5389273000000001, 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055436813000000001, 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2149 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198727+0000", 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198727+0000", 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198728+0000", 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198730+0000", 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6699999999999998e-06 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198739+0000", 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3499999999999997e-06 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198749+0000", 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.008e-05 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253726+0000", 2024-09-22T01:00:59.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054977005000000002 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253911+0000", 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000185451 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.254164+0000", 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025263 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2150 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198757+0000", 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5388970569999998, 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055676784, 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2150 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198757+0000", 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198757+0000", 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198758+0000", 2024-09-22T01:00:59.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198772+0000", 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4229999999999999e-05 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198789+0000", 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7187e-05 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:00:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198796+0000", 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8360000000000003e-06 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.254208+0000", 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055411929999999998 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.254299+0000", 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0420999999999994e-05 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:00:59.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.254434+0000", 2024-09-22T01:00:59.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013554999999999999 2024-09-22T01:00:59.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.828 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:00:59.828 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:00:59.828 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:00:59.829 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:00:59.830 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:00:59.825+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:00:59.930 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:00:59.928+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:00:59.944 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone3 intact 2024-09-22T01:01:00.000 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:00.001 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:00.001 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:00.001 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:00.017 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:00.030 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:00.028+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:00.131 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:00.128+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:00.138 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:00.138 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:00.138 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:00.138 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:00.138 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:00.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:00.165 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-22T01:01:00.231 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:00.228+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:00.235 INFO:teuthology.orchestra.run.smithi002.stdout:381 read 0x38f2269 thru 0x38f2ebc (0xc54 bytes) 2024-09-22T01:01:00.236 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xee23d7a 2024-09-22T01:01:00.254 INFO:teuthology.orchestra.run.smithi002.stdout:382 trunc from 0x42fde88 to 0xee23d7a 2024-09-22T01:01:00.272 INFO:teuthology.orchestra.run.smithi002.stdout:383 read 0xec6dbee thru 0xec72d47 (0x515a bytes) 2024-09-22T01:01:00.272 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.272 INFO:teuthology.orchestra.run.smithi002.stdout:385 read 0x58a9774 thru 0x58ab74d (0x1fda bytes) 2024-09-22T01:01:00.272 INFO:teuthology.orchestra.run.smithi002.stdout:386 write 0x9f18d58 thru 0x9f21db8 (0x9061 bytes) 2024-09-22T01:01:00.277 INFO:teuthology.orchestra.run.smithi002.stdout:387 write 0x995f904 thru 0x996074d (0xe4a bytes) 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2955 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.905888493, 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044281583999999999, 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2955 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378128+0000", 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378131+0000", 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.864e-06 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378137+0000", 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8020000000000003e-06 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378141+0000", 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422313+0000", 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044172553000000003 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422359+0000", 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6146000000000002e-05 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422409+0000", 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9576000000000001e-05 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2956 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3284992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9058588050000003, 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044364010000000002, 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2956 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378159+0000", 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8679999999999999e-06 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378164+0000", 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2499999999999997e-06 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378169+0000", 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3900000000000003e-06 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422438+0000", 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044268764000000002 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422478+0000", 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.066e-05 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422521+0000", 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2741999999999998e-05 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2957 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9058420839999997, 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044474276, 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2957 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:00.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1299999999999999e-07 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378175+0000", 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3039999999999999e-06 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378179+0000", 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.388e-06 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378188+0000", 2024-09-22T01:01:00.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0569999999999999e-06 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422550+0000", 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044362523000000001 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422597+0000", 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6493999999999999e-05 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422648+0000", 2024-09-22T01:01:00.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1297e-05 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2958 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9058220380000002, 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044568617999999997, 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2958 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.75e-07 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378196+0000", 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4959999999999999e-06 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378199+0000", 2024-09-22T01:01:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7969999999999998e-06 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378202+0000", 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.65e-06 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422678+0000", 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044475564000000002 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422718+0000", 2024-09-22T01:01:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0825000000000003e-05 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422762+0000", 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3810999999999998e-05 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2959 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9058097810000003, 2024-09-22T01:01:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044681193000000001, 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2959 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378207+0000", 2024-09-22T01:01:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.316e-06 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378211+0000", 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378213+0000", 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.588e-06 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422794+0000", 2024-09-22T01:01:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044580928999999998 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422840+0000", 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5231000000000001e-05 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422887+0000", 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7543999999999997e-05 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2960 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9057987360000004, 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044780915999999997, 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2960 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378218+0000", 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.032e-06 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378222+0000", 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4690000000000002e-06 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378224+0000", 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5040000000000001e-06 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422915+0000", 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044691036000000003 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422956+0000", 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0881999999999999e-05 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422998+0000", 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1782000000000001e-05 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2961 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.905788115, 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891911, 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2961 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378229+0000", 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.071e-06 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378232+0000", 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378235+0000", 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4609999999999998e-06 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423029+0000", 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044794498000000002 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423072+0000", 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2796000000000002e-05 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423120+0000", 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7574999999999998e-05 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2962 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.905777584, 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044991058, 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.314 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2962 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378239+0000", 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378243+0000", 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3459999999999998e-06 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378245+0000", 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6340000000000001e-06 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423149+0000", 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044903169999999999 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423188+0000", 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8974999999999999e-05 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423229+0000", 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1655e-05 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2963 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:00.318 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9057669720000003, 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104404000000001, 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378250+0000", 2024-09-22T01:01:00.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.001e-06 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378255+0000", 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7949999999999998e-06 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378264+0000", 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6129999999999993e-06 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423259+0000", 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044994348000000003 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423302+0000", 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3733000000000003e-05 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423353+0000", 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0689000000000002e-05 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.322 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2964 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9057466840000004, 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045200033000000001, 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2964 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378275+0000", 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3850000000000001e-06 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378279+0000", 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378281+0000", 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5129999999999999e-06 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423382+0000", 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045100385 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423422+0000", 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0265999999999997e-05 2024-09-22T01:01:00.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423469+0000", 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7110999999999997e-05 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2965 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9057241759999997, 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045300746000000003, 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.327 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2965 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6899999999999999e-07 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378294+0000", 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.951e-06 2024-09-22T01:01:00.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378298+0000", 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1239999999999998e-06 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378301+0000", 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.779e-06 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423497+0000", 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045196473000000001 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423544+0000", 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6622000000000002e-05 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423592+0000", 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8528e-05 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2966 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9057107779999996, 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045402558000000003, 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2966 2024-09-22T01:01:00.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:00.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378306+0000", 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2869999999999999e-06 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378310+0000", 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.258e-06 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378312+0000", 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7810000000000001e-06 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423622+0000", 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045309464000000001 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423663+0000", 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1310999999999998e-05 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423708+0000", 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4248000000000001e-05 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3013 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9056996809999998, 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045530159000000001, 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3013 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378317+0000", 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378321+0000", 2024-09-22T01:01:00.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3110000000000001e-06 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378323+0000", 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5600000000000001e-06 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423737+0000", 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045414035999999998 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423781+0000", 2024-09-22T01:01:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3279999999999999e-05 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423846+0000", 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5634000000000002e-05 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3014 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9056890229999999, 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045653157, 2024-09-22T01:01:00.340 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3014 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378715+0000", 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038801599999999998 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378719+0000", 2024-09-22T01:01:00.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4039999999999997e-06 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378731+0000", 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1755e-05 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378739+0000", 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.712e-06 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423876+0000", 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045137505000000001 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423918+0000", 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2339999999999998e-05 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423980+0000", 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1425000000000002e-05 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3015 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9052616489999998, 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045369596999999998, 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.345 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3015 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378755+0000", 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0100000000000003e-07 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378759+0000", 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378766+0000", 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.029e-06 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378769+0000", 2024-09-22T01:01:00.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7189999999999999e-06 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424013+0000", 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045243798000000002 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424059+0000", 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5725e-05 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424124+0000", 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4851999999999995e-05 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3016 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9052402500000003, 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045478877000000001, 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.349 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3016 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379160+0000", 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038423800000000002 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379164+0000", 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379173+0000", 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.439e-06 2024-09-22T01:01:00.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379178+0000", 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3909999999999997e-06 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424155+0000", 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044976781 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424198+0000", 2024-09-22T01:01:00.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769000000000003e-05 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424254+0000", 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6641000000000002e-05 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3017 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9048232110000001, 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045202985000000001, 2024-09-22T01:01:00.353 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3017 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9400000000000005e-07 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379198+0000", 2024-09-22T01:01:00.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379215+0000", 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6863000000000001e-05 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379219+0000", 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0749999999999999e-06 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424286+0000", 2024-09-22T01:01:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045067013000000003 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424332+0000", 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5531000000000002e-05 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424395+0000", 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3740999999999998e-05 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.357 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3018 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9047874580000004, 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045293472000000001, 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3018 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379430+0000", 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020146699999999999 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379435+0000", 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379443+0000", 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4819999999999994e-06 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379461+0000", 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.753e-05 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424424+0000", 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044962716999999999 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424464+0000", 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0824000000000001e-05 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424522+0000", 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7383999999999998e-05 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3019 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9045452019999995, 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045194026999999998, 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.362 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3019 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381690+0000", 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022191849999999998 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381696+0000", 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3860000000000004e-06 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381705+0000", 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2480000000000002e-06 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381712+0000", 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.579e-06 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424554+0000", 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042842327999999999 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424601+0000", 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6697999999999999e-05 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424665+0000", 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3602999999999994e-05 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:8990 3.a 3:51e071dd:::rbd_data.117cc87baa96.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.102999303, 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048973669999999997, 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:00.366 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 8990 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:00.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181019+0000", 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4889999999999998e-06 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181038+0000", 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9097e-05 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181066+0000", 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7514000000000001e-05 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181078+0000", 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2626000000000001e-05 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181196+0000", 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118208 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:00.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181388+0000", 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019203100000000001 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.188674+0000", 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072856120000000003 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229668+0000", 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040994282 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:00.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229764+0000", 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6036e-05 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229859+0000", 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5088999999999997e-05 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229990+0000", 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130686 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:00.372 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:00.372 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:00.372 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:00.374 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:00.375 INFO:teuthology.orchestra.run.smithi002.stdout:389 write 0x2d59a71 thru 0x2d65867 (0xbdf7 bytes) 2024-09-22T01:01:00.375 INFO:teuthology.orchestra.run.smithi002.stdout:391 write 0xb564781 thru 0xb56ea30 (0xa2b0 bytes) 2024-09-22T01:01:00.375 INFO:teuthology.orchestra.run.smithi002.stdout:392 punch from 0x8310d39 to 0x8313196, (0x245d bytes) 2024-09-22T01:01:00.375 INFO:teuthology.orchestra.run.smithi002.stdout:393 write 0xb1378ca thru 0xb13cdef (0x5526 bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:394 write 0xe1f2f42 thru 0xe1fe32b (0xb3ea bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:395 write 0xd72d9f6 thru 0xd72ffa6 (0x25b1 bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:400 punch from 0x4362d70 to 0x436616d, (0x33fd bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:403 writesame 0x7fa2a44 thru 0x7fa7cb2 data_size 0x1(0x526f bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:404 write 0xc6bfbe2 thru 0xc6cb449 (0xb868 bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:405 read 0xc520409 thru 0xc527a19 (0x7611 bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:406 read 0x46b95e0 thru 0x46c5946 (0xc367 bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:407 write 0xaa3c9b8 thru 0xaa4514d (0x8796 bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:409 write 0x909e878 thru 0x90a4455 (0x5bde bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:410 read 0x2735c4 thru 0x2737b9 (0x1f6 bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:413 punch from 0x741ea4a to 0x7421ac5, (0x307b bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:414 read 0x908fb66 thru 0x9099bf5 (0xa090 bytes) 2024-09-22T01:01:00.376 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.377 INFO:teuthology.orchestra.run.smithi002.stdout:416 punch from 0xa09e330 to 0xa0a044d, (0x211d bytes) 2024-09-22T01:01:00.377 INFO:teuthology.orchestra.run.smithi002.stdout:417 trunc from 0xee23d7a to 0xb595403 2024-09-22T01:01:00.377 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:00.372+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:418 punch from 0xb4e1c3d to 0xb4eaf13, (0x92d6 bytes) 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:420 read 0x84a0757 thru 0x84aa6b5 (0x9f5f bytes) 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:422 punch from 0x684a099 to 0x6857538, (0xd49f bytes) 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:423 read 0x641e330 thru 0x642ca5b (0xe72c bytes) 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:425 read 0x236b889 thru 0x237194b (0x60c3 bytes) 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:426 read 0x9a0fbdb thru 0x9a1a780 (0xaba6 bytes) 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.378 INFO:teuthology.orchestra.run.smithi002.stdout:429 writesame 0xeb3428 thru 0xebc22c data_size 0x1(0x8e05 bytes) 2024-09-22T01:01:00.413 INFO:teuthology.orchestra.run.smithi002.stdout:430 write 0xd446ce0 thru 0xd44f7eb (0x8b0c bytes) 2024-09-22T01:01:00.415 INFO:teuthology.orchestra.run.smithi002.stdout:431 punch from 0xafc10e6 to 0xafc6ed0, (0x5dea bytes) 2024-09-22T01:01:00.415 INFO:teuthology.orchestra.run.smithi002.stdout:432 write 0x4e9b7cd thru 0x4e9ec28 (0x345c bytes) 2024-09-22T01:01:00.421 INFO:teuthology.orchestra.run.smithi002.stdout:433 punch from 0xe8253f to 0xe8ec35, (0xc6f6 bytes) 2024-09-22T01:01:00.421 INFO:teuthology.orchestra.run.smithi002.stdout:435 write 0x1f6353b thru 0x1f67726 (0x41ec bytes) 2024-09-22T01:01:00.423 INFO:teuthology.orchestra.run.smithi002.stdout:436 write 0x13cb6cd thru 0x13d1520 (0x5e54 bytes) 2024-09-22T01:01:00.429 INFO:teuthology.orchestra.run.smithi002.stdout:437 read 0xa23161a thru 0xa23cf9d (0xb984 bytes) 2024-09-22T01:01:00.429 INFO:teuthology.orchestra.run.smithi002.stdout:438 write 0x8965953 thru 0x8966ba6 (0x1254 bytes) 2024-09-22T01:01:00.434 INFO:teuthology.orchestra.run.smithi002.stdout:440 punch from 0x66a0697 to 0x66ae682, (0xdfeb bytes) 2024-09-22T01:01:00.434 INFO:teuthology.orchestra.run.smithi002.stdout:441 write 0x2523e22 thru 0x2529240 (0x541f bytes) 2024-09-22T01:01:00.439 INFO:teuthology.orchestra.run.smithi002.stdout:442 read 0x144dc88 thru 0x144e534 (0x8ad bytes) 2024-09-22T01:01:00.439 INFO:teuthology.orchestra.run.smithi002.stdout:443 write 0x396e732 thru 0x397b29e (0xcb6d bytes) 2024-09-22T01:01:00.447 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.447 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.447 INFO:teuthology.orchestra.run.smithi002.stdout:447 read 0x43be776 thru 0x43c8f92 (0xa81d bytes) 2024-09-22T01:01:00.447 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.447 INFO:teuthology.orchestra.run.smithi002.stdout:449 write 0x7d3158a thru 0x7d37377 (0x5dee bytes) 2024-09-22T01:01:00.449 INFO:teuthology.orchestra.run.smithi002.stdout:450 punch from 0x976608c to 0x976e063, (0x7fd7 bytes) 2024-09-22T01:01:00.449 INFO:teuthology.orchestra.run.smithi002.stdout:451 read 0x3789098 thru 0x378fc20 (0x6b89 bytes) 2024-09-22T01:01:00.449 INFO:teuthology.orchestra.run.smithi002.stdout:452 read 0x66ee76c thru 0x66f9406 (0xac9b bytes) 2024-09-22T01:01:00.460 INFO:teuthology.orchestra.run.smithi002.stdout:453 writesame 0xee599fb thru 0xee62e4d data_size 0x3(0x9453 bytes) 2024-09-22T01:01:00.460 INFO:teuthology.orchestra.run.smithi002.stdout:457 read 0x2fa1968 thru 0x2fa682d (0x4ec6 bytes) 2024-09-22T01:01:00.460 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:00.460 INFO:teuthology.orchestra.run.smithi002.stdout:460 read 0xe7ff098 thru 0xe8072e8 (0x8251 bytes) 2024-09-22T01:01:00.460 INFO:teuthology.orchestra.run.smithi002.stdout:461 writesame 0x9d6661c thru 0x9d6cbb5 data_size 0x66(0x659a bytes) 2024-09-22T01:01:00.465 INFO:teuthology.orchestra.run.smithi002.stdout:462 read 0x36f6941 thru 0x36fc422 (0x5ae2 bytes) 2024-09-22T01:01:00.465 INFO:teuthology.orchestra.run.smithi002.stdout:463 writesame 0x76d9684 thru 0x76e005b data_size 0xc(0x69d8 bytes) 2024-09-22T01:01:00.472 INFO:teuthology.orchestra.run.smithi002.stdout:464 read 0x91585d5 thru 0x91652b4 (0xcce0 bytes) 2024-09-22T01:01:00.473 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:00.472+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:00.481 INFO:teuthology.orchestra.run.smithi002.stdout:465 trunc from 0xee62e4e to 0x9dd41cd 2024-09-22T01:01:00.495 INFO:teuthology.orchestra.run.smithi002.stdout:466 write 0xf5e02c thru 0xf60206 (0x21db bytes) 2024-09-22T01:01:00.498 INFO:teuthology.orchestra.run.smithi002.stdout:468 punch from 0x4af1cc8 to 0x4af8fd7, (0x730f bytes) 2024-09-22T01:01:00.498 INFO:teuthology.orchestra.run.smithi002.stdout:469 read 0x223759a thru 0x22409c8 (0x942f bytes) 2024-09-22T01:01:00.499 INFO:teuthology.orchestra.run.smithi002.stdout:472 write 0x310127b thru 0x3107600 (0x6386 bytes) 2024-09-22T01:01:00.505 INFO:teuthology.orchestra.run.smithi002.stdout:474 clone 5 order 18 su 65536 sc 14 2024-09-22T01:01:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:00.525 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:00.545 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:00.562 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:00.574 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:00.572+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:00.579 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:00.579 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:00.595 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:00.596 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:00.613 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:00.614 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:00.614 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:00.632 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:00.632 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:00.647 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:00.648 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:00.648 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:00.648 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:00.648 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:00.653 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:00.653 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:00.656 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:00.657 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:00.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:00.705 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:00.704+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:00.776 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:00.773+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:00.785 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:00.785 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:00.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:01:00.800 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:00.877 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:00.876+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:00.932 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone4 from 0x9dd41cd (overlap 0x42fde88) to 0x0 2024-09-22T01:01:00.978 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:00.976+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:00.991 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:00.991 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:01.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:01.080 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:01.077+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:01.112 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-22T01:01:01.181 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:01.177+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:01.182 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:01.183 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:01.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:01.252 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.252 INFO:teuthology.orchestra.run.smithi002.stdout:478 read 0x4c49544 thru 0x4c49ff9 (0xab6 bytes) 2024-09-22T01:01:01.253 INFO:teuthology.orchestra.run.smithi002.stdout:480 write 0x3de057a thru 0x3de54c0 (0x4f47 bytes) 2024-09-22T01:01:01.277 INFO:teuthology.orchestra.run.smithi002.stdout:481 writesame 0xcc1bc25 thru 0xcc22e80 data_size 0xd(0x725c bytes) 2024-09-22T01:01:01.277 INFO:teuthology.orchestra.run.smithi002.stdout:482 punch from 0x5944bce to 0x594dd8e, (0x91c0 bytes) 2024-09-22T01:01:01.277 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.277 INFO:teuthology.orchestra.run.smithi002.stdout:484 read 0x734e449 thru 0x7351c62 (0x381a bytes) 2024-09-22T01:01:01.277 INFO:teuthology.orchestra.run.smithi002.stdout:485 read 0x99ea8db thru 0x99f5b48 (0xb26e bytes) 2024-09-22T01:01:01.279 INFO:teuthology.orchestra.run.smithi002.stdout:486 read 0x68b0ab9 thru 0x68bdd17 (0xd25f bytes) 2024-09-22T01:01:01.281 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:01.277+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:01.287 INFO:teuthology.orchestra.run.smithi002.stdout:487 trunc from 0xcc22e81 to 0x70c62b1 2024-09-22T01:01:01.318 INFO:teuthology.orchestra.run.smithi002.stdout:488 read 0x459bf93 thru 0x45a113c (0x51aa bytes) 2024-09-22T01:01:01.318 INFO:teuthology.orchestra.run.smithi002.stdout:489 write 0x6ba70b5 thru 0x6bb0f68 (0x9eb4 bytes) 2024-09-22T01:01:01.327 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.330 INFO:teuthology.orchestra.run.smithi002.stdout:491 trunc from 0x70c62b1 to 0x501e3d0 2024-09-22T01:01:01.336 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:01.336 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:01.355 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:01.368 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.368 INFO:teuthology.orchestra.run.smithi002.stdout:493 read 0x43296ed thru 0x4333cb8 (0xa5cc bytes) 2024-09-22T01:01:01.381 INFO:teuthology.orchestra.run.smithi002.stdout:494 trunc from 0x501e3d0 to 0xe018ea9 2024-09-22T01:01:01.381 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:01.380+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:01.388 INFO:teuthology.orchestra.run.smithi002.stdout:495 read 0xf2f39c thru 0xf31051 (0x1cb6 bytes) 2024-09-22T01:01:01.389 INFO:teuthology.orchestra.run.smithi002.stdout:496 writesame 0xbf54b82 thru 0xbf571e8 data_size 0x3(0x2667 bytes) 2024-09-22T01:01:01.412 INFO:teuthology.orchestra.run.smithi002.stdout:497 trunc from 0xe018ea9 to 0x39a2e9a 2024-09-22T01:01:01.413 INFO:teuthology.orchestra.run.smithi119.stdout:get_pg_num_history pg_num_history pg_num_history(e25 pg_nums {1={15=1},2={18=8},3={22=32}} deleted_pools ) 2024-09-22T01:01:01.414 INFO:teuthology.orchestra.run.smithi119.stdout:Importing pgid 3.17 2024-09-22T01:01:01.414 INFO:teuthology.orchestra.run.smithi119.stdout:write_pg epoch 25 info 3.17( empty local-lis/les=22/23 n=0 ec=22/22 lis/c=22/22 les/c/f=23/23/0 sis=22) 2024-09-22T01:01:01.416 INFO:teuthology.orchestra.run.smithi119.stdout:Import successful 2024-09-22T01:01:01.443 INFO:teuthology.orchestra.run.smithi002.stdout:500 write 0x168e94e thru 0x1695886 (0x6f39 bytes) 2024-09-22T01:01:01.449 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.449 INFO:teuthology.orchestra.run.smithi002.stdout:505 writesame 0x1bdf8cc thru 0x1beccab data_size 0xa0(0xd3e0 bytes) 2024-09-22T01:01:01.477 INFO:teuthology.orchestra.run.smithi002.stdout:506 write 0xe2032f4 thru 0xe20cdb8 (0x9ac5 bytes) 2024-09-22T01:01:01.481 INFO:teuthology.orchestra.run.smithi002.stdout:508 punch from 0x2508db6 to 0x250f9ed, (0x6c37 bytes) 2024-09-22T01:01:01.482 INFO:teuthology.orchestra.run.smithi002.stdout:509 writesame 0x4ddc473 thru 0x4de493a data_size 0xe(0x84c8 bytes) 2024-09-22T01:01:01.482 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:01.480+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:01.486 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.486 INFO:teuthology.orchestra.run.smithi002.stdout:511 writesame 0xe8f46c thru 0xe958f4 data_size 0x17(0x6489 bytes) 2024-09-22T01:01:01.493 INFO:teuthology.orchestra.run.smithi002.stdout:512 read 0xb8592aa thru 0xb8684f7 (0xf24e bytes) 2024-09-22T01:01:01.499 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:01.499 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:01.503 INFO:teuthology.orchestra.run.smithi002.stdout:513 trunc from 0xe20cdb9 to 0x7718c6c 2024-09-22T01:01:01.517 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:01.521 INFO:teuthology.orchestra.run.smithi002.stdout:514 punch from 0x6040837 to 0x6041ac4, (0x128d bytes) 2024-09-22T01:01:01.532 INFO:teuthology.orchestra.run.smithi002.stdout:515 trunc from 0x7718c6c to 0xd41f265 2024-09-22T01:01:01.553 INFO:teuthology.orchestra.run.smithi002.stdout:518 write 0xed9daa5 thru 0xedaa569 (0xcac5 bytes) 2024-09-22T01:01:01.561 INFO:teuthology.orchestra.run.smithi002.stdout:519 read 0xa29a9cd thru 0xa29ee09 (0x443d bytes) 2024-09-22T01:01:01.561 INFO:teuthology.orchestra.run.smithi002.stdout:520 read 0x101c230 thru 0x1025422 (0x91f3 bytes) 2024-09-22T01:01:01.562 INFO:teuthology.orchestra.run.smithi002.stdout:521 write 0x4d385f5 thru 0x4d43d88 (0xb794 bytes) 2024-09-22T01:01:01.570 INFO:teuthology.orchestra.run.smithi002.stdout:522 write 0xd8b4be2 thru 0xd8c27ef (0xdc0e bytes) 2024-09-22T01:01:01.576 INFO:teuthology.orchestra.run.smithi002.stdout:523 write 0xd6611ba thru 0xd66dbdb (0xca22 bytes) 2024-09-22T01:01:01.582 INFO:teuthology.orchestra.run.smithi002.stdout:524 write 0x828aa9c thru 0x828f22f (0x4794 bytes) 2024-09-22T01:01:01.582 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:01.577+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:01.588 INFO:teuthology.orchestra.run.smithi002.stdout:525 punch from 0xcf0320f to 0xcf0d254, (0xa045 bytes) 2024-09-22T01:01:01.588 INFO:teuthology.orchestra.run.smithi002.stdout:526 punch from 0x2b993da to 0x2ba2cc6, (0x98ec bytes) 2024-09-22T01:01:01.588 INFO:teuthology.orchestra.run.smithi002.stdout:527 write 0x8b29b25 thru 0x8b36746 (0xcc22 bytes) 2024-09-22T01:01:01.595 INFO:teuthology.orchestra.run.smithi002.stdout:528 write 0x7d0d87c thru 0x7d13f8e (0x6713 bytes) 2024-09-22T01:01:01.603 INFO:teuthology.orchestra.run.smithi002.stdout:529 read 0x57d0550 thru 0x57d8645 (0x80f6 bytes) 2024-09-22T01:01:01.603 INFO:teuthology.orchestra.run.smithi002.stdout:530 punch from 0x918bed1 to 0x918d41d, (0x154c bytes) 2024-09-22T01:01:01.603 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.603 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.603 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.603 INFO:teuthology.orchestra.run.smithi002.stdout:536 write 0x5a39295 thru 0x5a3a611 (0x137d bytes) 2024-09-22T01:01:01.609 INFO:teuthology.orchestra.run.smithi002.stdout:538 read 0x568245d thru 0x5685f11 (0x3ab5 bytes) 2024-09-22T01:01:01.609 INFO:teuthology.orchestra.run.smithi002.stdout:541 punch from 0x536393f to 0x53661f3, (0x28b4 bytes) 2024-09-22T01:01:01.609 INFO:teuthology.orchestra.run.smithi002.stdout:545 punch from 0x695ae71 to 0x6967f33, (0xd0c2 bytes) 2024-09-22T01:01:01.609 INFO:teuthology.orchestra.run.smithi002.stdout:546 write 0x16782b5 thru 0x167fa53 (0x779f bytes) 2024-09-22T01:01:01.615 INFO:teuthology.orchestra.run.smithi002.stdout:547 punch from 0x80287d8 to 0x80343aa, (0xbbd2 bytes) 2024-09-22T01:01:01.615 INFO:teuthology.orchestra.run.smithi002.stdout:548 read 0xc08e0ae thru 0xc09bf38 (0xde8b bytes) 2024-09-22T01:01:01.615 INFO:teuthology.orchestra.run.smithi002.stdout:549 writesame 0xe836d3 thru 0xe84dd6 data_size 0x6(0x1704 bytes) 2024-09-22T01:01:01.618 INFO:teuthology.orchestra.run.smithi002.stdout:552 read 0x275833 thru 0x2784a9 (0x2c77 bytes) 2024-09-22T01:01:01.618 INFO:teuthology.orchestra.run.smithi002.stdout:554 read 0xaac2c77 thru 0xaac5a13 (0x2d9d bytes) 2024-09-22T01:01:01.628 INFO:teuthology.orchestra.run.smithi002.stdout:555 trunc from 0xedaa56a to 0x88dccf3 2024-09-22T01:01:01.662 INFO:teuthology.orchestra.run.smithi002.stdout:556 trunc from 0x88dccf3 to 0x4adac7 2024-09-22T01:01:01.668 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:01.669 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:01.683 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:01.680+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:01.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:01.727 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.751 INFO:teuthology.orchestra.run.smithi002.stdout:559 write 0x7633599 thru 0x763b215 (0x7c7d bytes) 2024-09-22T01:01:01.775 INFO:teuthology.orchestra.run.smithi002.stdout:560 write 0xbb07060 thru 0xbb13fb4 (0xcf55 bytes) 2024-09-22T01:01:01.781 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:01:01.783 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:01.780+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:01.798 INFO:teuthology.orchestra.run.smithi002.stdout:561 trunc from 0xbb13fb5 to 0x86914b1 2024-09-22T01:01:01.834 INFO:teuthology.orchestra.run.smithi002.stdout:562 writesame 0xde17aa3 thru 0xde2591d data_size 0x5(0xde7b bytes) 2024-09-22T01:01:01.834 INFO:teuthology.orchestra.run.smithi002.stdout:563 read 0x46c50ed thru 0x46ced4c (0x9c60 bytes) 2024-09-22T01:01:01.834 INFO:teuthology.orchestra.run.smithi002.stdout:565 write 0xb92e2d1 thru 0xb930ce4 (0x2a14 bytes) 2024-09-22T01:01:01.843 INFO:teuthology.orchestra.run.smithi002.stdout:566 read 0x7758f68 thru 0x7760fe5 (0x807e bytes) 2024-09-22T01:01:01.843 INFO:teuthology.orchestra.run.smithi002.stdout:567 read 0x24c8016 thru 0x24d05e0 (0x85cb bytes) 2024-09-22T01:01:01.843 INFO:teuthology.orchestra.run.smithi002.stdout:568 read 0x50396ea thru 0x5048d5a (0xf671 bytes) 2024-09-22T01:01:01.843 INFO:teuthology.orchestra.run.smithi002.stdout:571 writesame 0x5e152c5 thru 0x5e17ced data_size 0x1(0x2a29 bytes) 2024-09-22T01:01:01.860 INFO:teuthology.orchestra.run.smithi002.stdout:573 read 0x8b63828 thru 0x8b6f42b (0xbc04 bytes) 2024-09-22T01:01:01.861 INFO:teuthology.orchestra.run.smithi002.stdout:574 read 0xb87bc7b thru 0xb88a792 (0xeb18 bytes) 2024-09-22T01:01:01.861 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.861 INFO:teuthology.orchestra.run.smithi002.stdout:576 write 0x15f366e thru 0x1600092 (0xca25 bytes) 2024-09-22T01:01:01.869 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:01.869 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:01.884 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:01.880+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:01.887 INFO:teuthology.orchestra.run.smithi002.stdout:578 trunc from 0xde2591e to 0x2e5ecee 2024-09-22T01:01:01.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:01.932 INFO:teuthology.orchestra.run.smithi002.stdout:579 write 0xb6ad4ba thru 0xb6b228f (0x4dd6 bytes) 2024-09-22T01:01:01.940 INFO:teuthology.orchestra.run.smithi002.stdout:581 punch from 0x4a8e232 to 0x4a96f1f, (0x8ced bytes) 2024-09-22T01:01:01.940 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:01.941 INFO:teuthology.orchestra.run.smithi002.stdout:584 punch from 0x98c441d to 0x98d1745, (0xd328 bytes) 2024-09-22T01:01:01.952 INFO:teuthology.orchestra.run.smithi002.stdout:585 write 0xd4a3131 thru 0xd4b1a1c (0xe8ec bytes) 2024-09-22T01:01:01.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /var/log/ceph/exp.3.17.5 2024-09-22T01:01:01.974 INFO:teuthology.orchestra.run.smithi002.stdout:586 trunc from 0xd4b1a1d to 0x42ab1a3 2024-09-22T01:01:01.984 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:01.980+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:01.989 INFO:teuthology.orchestra.run.smithi002.stdout:587 punch from 0x118710d to 0x11942c8, (0xd1bb bytes) 2024-09-22T01:01:01.989 INFO:teuthology.orchestra.run.smithi002.stdout:588 read 0x90d8 thru 0x13d31 (0xac5a bytes) 2024-09-22T01:01:01.995 INFO:teuthology.orchestra.run.smithi002.stdout:589 trunc from 0x42ab1a3 to 0xa98956 2024-09-22T01:01:02.008 INFO:teuthology.orchestra.run.smithi002.stdout:590 read 0xa34f6c thru 0xa4494b (0xf9e0 bytes) 2024-09-22T01:01:02.022 INFO:teuthology.orchestra.run.smithi002.stdout:591 trunc from 0xa98956 to 0x9c8e828 2024-09-22T01:01:02.030 INFO:teuthology.orchestra.run.smithi002.stdout:592 writesame 0x7d5f542 thru 0x7d62d10 data_size 0xd(0x37cf bytes) 2024-09-22T01:01:02.038 INFO:teuthology.orchestra.run.smithi002.stdout:593 trunc from 0x9c8e828 to 0xa55b07b 2024-09-22T01:01:02.046 INFO:teuthology.orchestra.run.smithi002.stdout:594 writesame 0x29cb0e9 thru 0x29cf73c data_size 0xe(0x4654 bytes) 2024-09-22T01:01:02.051 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.052 INFO:teuthology.orchestra.run.smithi002.stdout:596 read 0x8331f41 thru 0x83322b3 (0x373 bytes) 2024-09-22T01:01:02.052 INFO:teuthology.orchestra.run.smithi002.stdout:597 write 0x9415ca thru 0x94667d (0x50b4 bytes) 2024-09-22T01:01:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:599 writesame 0x9d444bb thru 0x9d49902 data_size 0x1f(0x5448 bytes) 2024-09-22T01:01:02.071 INFO:teuthology.orchestra.run.smithi002.stdout:600 write 0xab415db thru 0xab4827f (0x6ca5 bytes) 2024-09-22T01:01:02.075 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:02.075 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:02.076 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.085 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:02.084+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:02.091 INFO:teuthology.orchestra.run.smithi002.stdout:602 write 0xd7a3693 thru 0xd7b355b (0xfec9 bytes) 2024-09-22T01:01:02.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:02.099 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.099 INFO:teuthology.orchestra.run.smithi002.stdout:604 write 0x387458e thru 0x3884573 (0xffe6 bytes) 2024-09-22T01:01:02.105 INFO:teuthology.orchestra.run.smithi002.stdout:608 write 0x4a1a44b thru 0x4a21ade (0x7694 bytes) 2024-09-22T01:01:02.112 INFO:teuthology.orchestra.run.smithi002.stdout:609 writesame 0x3bdfa8b thru 0x3bebad2 data_size 0x38(0xc048 bytes) 2024-09-22T01:01:02.120 INFO:teuthology.orchestra.run.smithi002.stdout:612 punch from 0x153460c to 0x15372d8, (0x2ccc bytes) 2024-09-22T01:01:02.120 INFO:teuthology.orchestra.run.smithi002.stdout:614 writesame 0xa1e886d thru 0xa1ea3f1 data_size 0x1(0x1b85 bytes) 2024-09-22T01:01:02.132 INFO:teuthology.orchestra.run.smithi002.stdout:615 punch from 0xb19efb4 to 0xb1aca9e, (0xdaea bytes) 2024-09-22T01:01:02.139 INFO:teuthology.orchestra.run.smithi002.stdout:616 trunc from 0xd7b355c to 0xa1e7e8c 2024-09-22T01:01:02.160 INFO:teuthology.orchestra.run.smithi002.stdout:617 write 0x66373a2 thru 0x6645620 (0xe27f bytes) 2024-09-22T01:01:02.181 INFO:teuthology.orchestra.run.smithi002.stdout:618 trunc from 0xa1e7e8c to 0x4619eb9 2024-09-22T01:01:02.186 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:02.181+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:02.200 INFO:teuthology.orchestra.run.smithi002.stdout:619 read 0x318b495 thru 0x3191279 (0x5de5 bytes) 2024-09-22T01:01:02.200 INFO:teuthology.orchestra.run.smithi002.stdout:620 punch from 0x19886af to 0x1992c5a, (0xa5ab bytes) 2024-09-22T01:01:02.200 INFO:teuthology.orchestra.run.smithi002.stdout:621 punch from 0x2e1b7b4 to 0x2e297c0, (0xe00c bytes) 2024-09-22T01:01:02.227 INFO:teuthology.orchestra.run.smithi002.stdout:622 write 0xd73958c thru 0xd7456ef (0xc164 bytes) 2024-09-22T01:01:02.236 INFO:teuthology.orchestra.run.smithi002.stdout:624 trunc from 0xd7456f0 to 0xc0b5451 2024-09-22T01:01:02.236 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:02.236 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:02.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:02.257 INFO:teuthology.orchestra.run.smithi002.stdout:625 read 0xc7a325 thru 0xc87ec1 (0xdb9d bytes) 2024-09-22T01:01:02.257 INFO:teuthology.orchestra.run.smithi002.stdout:627 write 0x42a5062 thru 0x42a531c (0x2bb bytes) 2024-09-22T01:01:02.267 INFO:teuthology.orchestra.run.smithi002.stdout:628 trunc from 0xc0b5451 to 0xe60f393 2024-09-22T01:01:02.276 INFO:teuthology.orchestra.run.smithi002.stdout:629 write 0x6eec64d thru 0x6ef2b8b (0x653f bytes) 2024-09-22T01:01:02.283 INFO:teuthology.orchestra.run.smithi002.stdout:630 punch from 0x8cb146b to 0x8cbd44a, (0xbfdf bytes) 2024-09-22T01:01:02.283 INFO:teuthology.orchestra.run.smithi002.stdout:631 punch from 0x2b81669 to 0x2b8b9d6, (0xa36d bytes) 2024-09-22T01:01:02.283 INFO:teuthology.orchestra.run.smithi002.stdout:632 writesame 0x84f965d thru 0x84fa828 data_size 0x11(0x11cc bytes) 2024-09-22T01:01:02.287 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:02.281+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:02.289 INFO:teuthology.orchestra.run.smithi002.stdout:633 read 0x682a095 thru 0x6832e98 (0x8e04 bytes) 2024-09-22T01:01:02.289 INFO:teuthology.orchestra.run.smithi002.stdout:634 writesame 0x3d1cdb3 thru 0x3d2735e data_size 0x2e(0xa5ac bytes) 2024-09-22T01:01:02.296 INFO:teuthology.orchestra.run.smithi002.stdout:635 read 0xcc19df5 thru 0xcc22260 (0x846c bytes) 2024-09-22T01:01:02.319 INFO:teuthology.orchestra.run.smithi002.stdout:636 trunc from 0xe60f393 to 0x25d6218 2024-09-22T01:01:02.341 INFO:teuthology.orchestra.run.smithi002.stdout:638 writesame 0x1e84fa5 thru 0x1e8670a data_size 0x5(0x1766 bytes) 2024-09-22T01:01:02.349 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.350 INFO:teuthology.orchestra.run.smithi002.stdout:640 trunc from 0x25d6218 to 0x382bf49 2024-09-22T01:01:02.383 INFO:teuthology.orchestra.run.smithi002.stdout:643 write 0x7950640 thru 0x7953372 (0x2d33 bytes) 2024-09-22T01:01:02.387 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:02.384+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:02.389 INFO:teuthology.orchestra.run.smithi002.stdout:644 punch from 0x1ea3eb2 to 0x1eb3e87, (0xffd5 bytes) 2024-09-22T01:01:02.390 INFO:teuthology.orchestra.run.smithi002.stdout:650 read 0x4ad8f90 thru 0x4ad9c7e (0xcef bytes) 2024-09-22T01:01:02.414 INFO:teuthology.orchestra.run.smithi002.stdout:651 writesame 0xdd73fae thru 0xdd7847a data_size 0x39(0x44cd bytes) 2024-09-22T01:01:02.414 INFO:teuthology.orchestra.run.smithi002.stdout:652 read 0xbe69f81 thru 0xbe6f0cc (0x514c bytes) 2024-09-22T01:01:02.414 INFO:teuthology.orchestra.run.smithi002.stdout:653 writesame 0x34f3286 thru 0x34fb6d7 data_size 0x1(0x8452 bytes) 2024-09-22T01:01:02.415 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:02.415 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:02.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:02.456 INFO:teuthology.orchestra.run.smithi002.stdout:654 write 0xd499573 thru 0xd4a2710 (0x919e bytes) 2024-09-22T01:01:02.465 INFO:teuthology.orchestra.run.smithi002.stdout:655 punch from 0x67704fe to 0x677f312, (0xee14 bytes) 2024-09-22T01:01:02.465 INFO:teuthology.orchestra.run.smithi002.stdout:658 punch from 0x66dad47 to 0x66e74ef, (0xc7a8 bytes) 2024-09-22T01:01:02.465 INFO:teuthology.orchestra.run.smithi002.stdout:659 read 0xaca6afc thru 0xacacba2 (0x60a7 bytes) 2024-09-22T01:01:02.465 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.465 INFO:teuthology.orchestra.run.smithi002.stdout:661 write 0x442796b thru 0x4430ccb (0x9361 bytes) 2024-09-22T01:01:02.473 INFO:teuthology.orchestra.run.smithi002.stdout:662 trunc from 0xdd7847b to 0xe5308e7 2024-09-22T01:01:02.481 INFO:teuthology.orchestra.run.smithi002.stdout:663 read 0xc10d136 thru 0xc11b668 (0xe533 bytes) 2024-09-22T01:01:02.481 INFO:teuthology.orchestra.run.smithi002.stdout:664 writesame 0xc9418cb thru 0xc9434d4 data_size 0x25(0x1c0a bytes) 2024-09-22T01:01:02.487 INFO:teuthology.orchestra.run.smithi002.stdout:666 read 0x3abaac1 thru 0x3ac0992 (0x5ed2 bytes) 2024-09-22T01:01:02.487 INFO:teuthology.orchestra.run.smithi002.stdout:667 read 0x507b925 thru 0x508b0eb (0xf7c7 bytes) 2024-09-22T01:01:02.487 INFO:teuthology.orchestra.run.smithi002.stdout:668 read 0xe4ac28 thru 0xe4e661 (0x3a3a bytes) 2024-09-22T01:01:02.487 INFO:teuthology.orchestra.run.smithi002.stdout:669 read 0x211123 thru 0x21ab3c (0x9a1a bytes) 2024-09-22T01:01:02.488 INFO:teuthology.orchestra.run.smithi002.stdout:670 writesame 0x811c25f thru 0x8124ae5 data_size 0x1(0x8887 bytes) 2024-09-22T01:01:02.488 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:02.484+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:02.520 INFO:teuthology.orchestra.run.smithi002.stdout:671 trunc from 0xe5308e7 to 0x51fc057 2024-09-22T01:01:02.544 INFO:teuthology.orchestra.run.smithi002.stdout:672 trunc from 0x51fc057 to 0x3d67561 2024-09-22T01:01:02.562 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.562 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.562 INFO:teuthology.orchestra.run.smithi002.stdout:675 read 0xed403a thru 0xedabbe (0x6b85 bytes) 2024-09-22T01:01:02.562 INFO:teuthology.orchestra.run.smithi002.stdout:676 read 0x517f76 thru 0x51841b (0x4a6 bytes) 2024-09-22T01:01:02.562 INFO:teuthology.orchestra.run.smithi002.stdout:677 read 0x9b6ea3 thru 0x9c2263 (0xb3c1 bytes) 2024-09-22T01:01:02.583 INFO:teuthology.orchestra.run.smithi002.stdout:678 writesame 0xa60867b thru 0xa60cdcb data_size 0x1(0x4751 bytes) 2024-09-22T01:01:02.586 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:02.586 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:02.590 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:02.588+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:02.593 INFO:teuthology.orchestra.run.smithi002.stdout:679 trunc from 0xa60cdcc to 0x49421b0 2024-09-22T01:01:02.606 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:02.607 INFO:teuthology.orchestra.run.smithi002.stdout:680 write 0x1c5fb84 thru 0x1c62333 (0x27b0 bytes) 2024-09-22T01:01:02.634 INFO:teuthology.orchestra.run.smithi002.stdout:682 writesame 0x8eb58a1 thru 0x8eba215 data_size 0x1(0x4975 bytes) 2024-09-22T01:01:02.634 INFO:teuthology.orchestra.run.smithi002.stdout:684 writesame 0x41b816b thru 0x41c52fe data_size 0x66(0xd194 bytes) 2024-09-22T01:01:02.643 INFO:teuthology.orchestra.run.smithi002.stdout:685 read 0x88b86a6 thru 0x88b8b0d (0x468 bytes) 2024-09-22T01:01:02.643 INFO:teuthology.orchestra.run.smithi002.stdout:686 read 0x5821c86 thru 0x58244fd (0x2878 bytes) 2024-09-22T01:01:02.643 INFO:teuthology.orchestra.run.smithi002.stdout:689 read 0x377408e thru 0x377a58f (0x6502 bytes) 2024-09-22T01:01:02.643 INFO:teuthology.orchestra.run.smithi002.stdout:691 write 0x1fe4da3 thru 0x1fe7297 (0x24f5 bytes) 2024-09-22T01:01:02.658 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:02.666 INFO:teuthology.orchestra.run.smithi002.stdout:694 writesame 0xd15b896 thru 0xd16824b data_size 0x1(0xc9b6 bytes) 2024-09-22T01:01:02.666 INFO:teuthology.orchestra.run.smithi002.stdout:695 read 0x6bd8db7 thru 0x6bda0ee (0x1338 bytes) 2024-09-22T01:01:02.666 INFO:teuthology.orchestra.run.smithi002.stdout:696 punch from 0xc35607f to 0xc35d1cb, (0x714c bytes) 2024-09-22T01:01:02.666 INFO:teuthology.orchestra.run.smithi002.stdout:697 read 0xb0f6c58 thru 0xb0fdf16 (0x72bf bytes) 2024-09-22T01:01:02.667 INFO:teuthology.orchestra.run.smithi002.stdout:698 writesame 0xb469941 thru 0xb46ca7f data_size 0x1(0x313f bytes) 2024-09-22T01:01:02.679 INFO:teuthology.orchestra.run.smithi002.stdout:700 punch from 0x1341947 to 0x134e4f2, (0xcbab bytes) 2024-09-22T01:01:02.679 INFO:teuthology.orchestra.run.smithi002.stdout:703 read 0xbf7b408 thru 0xbf8200e (0x6c07 bytes) 2024-09-22T01:01:02.680 INFO:teuthology.orchestra.run.smithi002.stdout:704 trunc from 0xd16824c to 0xc9b5336 2024-09-22T01:01:02.688 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.689 INFO:teuthology.orchestra.run.smithi002.stdout:707 punch from 0x7c63528 to 0x7c64b7e, (0x1656 bytes) 2024-09-22T01:01:02.689 INFO:teuthology.orchestra.run.smithi002.stdout:708 punch from 0x8fef31 to 0x90879c, (0x986b bytes) 2024-09-22T01:01:02.689 INFO:teuthology.orchestra.run.smithi002.stdout:710 read 0x3b1f982 thru 0x3b24f83 (0x5602 bytes) 2024-09-22T01:01:02.689 INFO:teuthology.orchestra.run.smithi002.stdout:711 read 0x25954de thru 0x25964c8 (0xfeb bytes) 2024-09-22T01:01:02.689 INFO:teuthology.orchestra.run.smithi002.stdout:714 read 0x6e6b88b thru 0x6e7aba8 (0xf31e bytes) 2024-09-22T01:01:02.689 INFO:teuthology.orchestra.run.smithi002.stdout:717 write 0x5e0c92a thru 0x5e14931 (0x8008 bytes) 2024-09-22T01:01:02.690 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:02.685+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:02.697 INFO:teuthology.orchestra.run.smithi002.stdout:718 writesame 0xc317482 thru 0xc326953 data_size 0x1(0xf4d2 bytes) 2024-09-22T01:01:02.763 INFO:teuthology.orchestra.run.smithi002.stdout:719 trunc from 0xc9b5336 to 0x6646ba1 2024-09-22T01:01:02.772 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:02.772 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:02.782 INFO:teuthology.orchestra.run.smithi002.stdout:720 read 0x805ae1 thru 0x80dafc (0x801c bytes) 2024-09-22T01:01:02.783 INFO:teuthology.orchestra.run.smithi002.stdout:721 punch from 0x2ee21a7 to 0x2eeb40a, (0x9263 bytes) 2024-09-22T01:01:02.783 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:02.783 INFO:teuthology.orchestra.run.smithi002.stdout:723 writesame 0x56b5743 thru 0x56b7fd9 data_size 0x1(0x2897 bytes) 2024-09-22T01:01:02.791 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:02.791 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:02.788+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:02.800 INFO:teuthology.orchestra.run.smithi002.stdout:724 write 0x2c73830 thru 0x2c7aaa1 (0x7272 bytes) 2024-09-22T01:01:02.804 INFO:teuthology.orchestra.run.smithi002.stdout:725 clone 6 order 22 su 262144 sc 13 2024-09-22T01:01:02.813 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:02.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:02.830 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:02.846 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:02.863 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:02.863 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:02.884 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:02.884 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:02.884 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:02.892 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:02.889+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:02.905 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:02.906 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1182 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8554399989999997, 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054296337, 2024-09-22T01:01:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1182 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2399999999999999e-07 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061556+0000", 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3700000000000002e-06 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061563+0000", 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0149999999999998e-06 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061568+0000", 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115682+0000", 2024-09-22T01:01:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054113513000000002 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115767+0000", 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4670000000000001e-05 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115850+0000", 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3264999999999996e-05 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1183 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8554182629999998, 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054497446999999997, 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1183 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061576+0000", 2024-09-22T01:01:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.65e-07 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061585+0000", 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4960000000000002e-06 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061593+0000", 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5340000000000002e-06 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061603+0000", 2024-09-22T01:01:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0213e-05 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115905+0000", 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054302187000000002 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115985+0000", 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9429000000000001e-05 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116073+0000", 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8222999999999999e-05 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1184 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8553829339999997, 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054590382999999999, 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1184 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061613+0000", 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061620+0000", 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.781e-06 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061625+0000", 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8999999999999997e-06 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116113+0000", 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054488414999999998 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116154+0000", 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0552000000000001e-05 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116201+0000", 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1185 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8553616640000001, 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054816747999999998, 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1185 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.34e-07 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061634+0000", 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.996e-06 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061641+0000", 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.455e-06 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061645+0000", 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6979999999999997e-06 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116274+0000", 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054628659000000003 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116363+0000", 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8721000000000007e-05 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116449+0000", 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5785000000000006e-05 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1186 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8553415749999997, 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055030429999999998, 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1186 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061662+0000", 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1700000000000003e-06 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061669+0000", 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4109999999999999e-06 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061674+0000", 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999999e-06 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116496+0000", 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054821684000000002 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116585+0000", 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9276000000000004e-05 2024-09-22T01:01:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116682+0000", 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7386000000000006e-05 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1187 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8553115560000002, 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055229227999999998, 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1187 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061685+0000", 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.289e-06 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061691+0000", 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7710000000000001e-06 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061702+0000", 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0443e-05 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116727+0000", 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055024977000000003 2024-09-22T01:01:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116811+0000", 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4524000000000006e-05 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116911+0000", 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9802000000000004e-05 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1282 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8543349720000002, 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054493779999999999, 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1282 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062660+0000", 2024-09-22T01:01:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6500000000000008e-07 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062664+0000", 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2080000000000002e-06 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062676+0000", 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.216e-05 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062684+0000", 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.576e-06 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116948+0000", 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054264616000000002 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117035+0000", 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7331999999999997e-05 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117152+0000", 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011692300000000001 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1283 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8542971100000001, 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054680980999999997, 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1283 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062697+0000", 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3900000000000005e-07 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062700+0000", 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062708+0000", 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8019999999999992e-06 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062713+0000", 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609999999999998e-06 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117187+0000", 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054473426999999998 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117270+0000", 2024-09-22T01:01:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3353e-05 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117377+0000", 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010757899999999999 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1284 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.854271368, 2024-09-22T01:01:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054881757000000003, 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1284 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062723+0000", 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062725+0000", 2024-09-22T01:01:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062732+0000", 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062737+0000", 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2580000000000003e-06 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117413+0000", 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054675834 2024-09-22T01:01:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117498+0000", 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4753000000000007e-05 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117604+0000", 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106363 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1285 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.854249298, 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055095997000000001, 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1285 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062745+0000", 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062747+0000", 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0760000000000001e-06 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062754+0000", 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.844e-06 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062759+0000", 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9799999999999998e-06 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117639+0000", 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054880802999999999 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117723+0000", 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3746999999999996e-05 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117840+0000", 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011718900000000001 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1286 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8542150490000004, 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055282537, 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1286 2024-09-22T01:01:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062782+0000", 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.835e-06 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062790+0000", 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.055e-06 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062796+0000", 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117873+0000", 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055077470000000003 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117951+0000", 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118061+0000", 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110027 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1287 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8541898230000005, 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055478658, 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1287 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062806+0000", 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.142e-06 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062813+0000", 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.613e-06 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062818+0000", 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118096+0000", 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055277739999999999 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118180+0000", 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4795000000000004e-05 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118282+0000", 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101974 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1288 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.969 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8541658300000003, 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055678514999999998, 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1288 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1199999999999998e-07 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062830+0000", 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.114e-06 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062838+0000", 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8599999999999993e-06 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062843+0000", 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118315+0000", 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055471883999999999 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118402+0000", 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6992999999999999e-05 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118506+0000", 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104238 2024-09-22T01:01:02.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1289 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8541326429999998, 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055873772000000002, 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1289 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063911+0000", 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010504500000000001 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063923+0000", 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1805e-05 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063947+0000", 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3519999999999998e-05 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063964+0000", 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6966e-05 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118539+0000", 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054574947999999998 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118631+0000", 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2238999999999996e-05 2024-09-22T01:01:02.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118735+0000", 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103844 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1290 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8529898170000001, 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054954785999999999, 2024-09-22T01:01:02.978 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1290 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064013+0000", 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064020+0000", 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0310000000000001e-06 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064044+0000", 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4340000000000001e-05 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064081+0000", 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6720000000000001e-05 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118769+0000", 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054687604000000001 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118852+0000", 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118959+0000", 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106969 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.982 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1291 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8528895429999999, 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055078496999999997, 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1291 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064105+0000", 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.268e-06 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064154+0000", 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9122999999999997e-05 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064174+0000", 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0057000000000001e-05 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064189+0000", 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4083000000000001e-05 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118992+0000", 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054803604999999998 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119079+0000", 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7109999999999998e-05 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119183+0000", 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103251 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1292 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.852783241, 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055189561999999998, 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1292 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065373+0000", 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001162419 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065382+0000", 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6979999999999993e-06 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065403+0000", 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0676e-05 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065418+0000", 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.468e-05 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119218+0000", 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053799877000000003 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119301+0000", 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2910000000000004e-05 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119400+0000", 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9302000000000006e-05 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.990 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1295 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8514239809999999, 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054395285000000002, 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.991 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1295 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065571+0000", 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065575+0000", 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065590+0000", 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.456e-05 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065616+0000", 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5935000000000002e-05 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119786+0000", 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054170124 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119862+0000", 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5833000000000004e-05 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119965+0000", 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103281 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1296 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1327104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8513616739999996, 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054554510000000001, 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1296 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:02.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066090+0000", 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045860600000000002 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066104+0000", 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3883000000000001e-05 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066125+0000", 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0392000000000001e-05 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066139+0000", 2024-09-22T01:01:02.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3925e-05 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120000+0000", 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053860906 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120084+0000", 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4875000000000003e-05 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120186+0000", 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010192300000000001 2024-09-22T01:01:02.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1297 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8508326139999998, 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054246075999999997, 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1297 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:02.999 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066728+0000", 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056672199999999997 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:03.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066733+0000", 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7400000000000001e-06 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066745+0000", 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.166e-05 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066753+0000", 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1620000000000008e-06 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120220+0000", 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05346687 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120303+0000", 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3354000000000002e-05 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120407+0000", 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103568 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:03.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:03.003 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:03.003 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:03.003 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:03.004 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:03.004 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:03.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:03.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:03.005 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:03.000+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:03.104 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:03.101+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:03.136 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:03.136 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:03.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:03.205 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:03.204+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:03.306 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:03.301+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:03.316 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:03.317 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:03.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:03.407 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:03.404+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:03.501 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:03.501 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:03.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:03.501 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:03.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.501 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2131 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2416640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.501 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198060+0000", 2024-09-22T01:01:03.501 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3018236269999992, 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047864156999999997, 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2131 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198060+0000", 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198060+0000", 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198061+0000", 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.23e-07 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198064+0000", 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198073+0000", 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4530000000000008e-06 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198080+0000", 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8530000000000003e-06 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.245415+0000", 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047334721000000003 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.245604+0000", 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018855 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.245924+0000", 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00032068299999999999 2024-09-22T01:01:03.505 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2132 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198089+0000", 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3017943699999996, 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048233273, 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.506 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2132 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198089+0000", 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198089+0000", 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198090+0000", 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4e-07 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198093+0000", 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7659999999999999e-06 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198101+0000", 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4700000000000002e-06 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198108+0000", 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6100000000000002e-06 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246031+0000", 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047922861999999997 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246137+0000", 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105715 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246323+0000", 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018600999999999999 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2133 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198117+0000", 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3017670999999993, 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048784292999999999, 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2133 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198117+0000", 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198117+0000", 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198117+0000", 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1600000000000001e-07 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198120+0000", 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6740000000000001e-06 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198128+0000", 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-22T01:01:03.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198153+0000", 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4583000000000002e-05 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246412+0000", 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048258425000000001 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246595+0000", 2024-09-22T01:01:03.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018392899999999999 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.246901+0000", 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030544300000000002 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2134 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198163+0000", 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3017211480000004, 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049133563999999998, 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2134 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198163+0000", 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198163+0000", 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198163+0000", 2024-09-22T01:01:03.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198166+0000", 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8729999999999999e-06 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198179+0000", 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2466999999999999e-05 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198186+0000", 2024-09-22T01:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9399999999999996e-06 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247008+0000", 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048822881999999998 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247116+0000", 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107316 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247296+0000", 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000180366 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2135 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2777088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198194+0000", 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3016892779999996, 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049682965000000003, 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.519 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2135 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198194+0000", 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198194+0000", 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198195+0000", 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198198+0000", 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8499999999999998e-06 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198207+0000", 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6260000000000001e-06 2024-09-22T01:01:03.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198215+0000", 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1799999999999996e-06 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247384+0000", 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049169599000000001 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247571+0000", 2024-09-22T01:01:03.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000187164 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247877+0000", 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030589899999999999 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2136 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198227+0000", 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3016568910000004, 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049974031000000002, 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2136 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198227+0000", 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198227+0000", 2024-09-22T01:01:03.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198227+0000", 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198230+0000", 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6570000000000001e-06 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198239+0000", 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.653e-06 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198245+0000", 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6800000000000004e-06 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.247943+0000", 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049697386000000003 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248033+0000", 2024-09-22T01:01:03.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0062000000000006e-05 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248201+0000", 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016796700000000001 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2137 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 2957312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198254+0000", 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3016295509999996, 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050520378999999997, 2024-09-22T01:01:03.527 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2137 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198254+0000", 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198254+0000", 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198255+0000", 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198257+0000", 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6299999999999998e-06 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198278+0000", 2024-09-22T01:01:03.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0919000000000001e-05 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198288+0000", 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.217e-06 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248284+0000", 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049996051 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248470+0000", 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018665399999999999 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248775+0000", 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000304285 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2138 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3047424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198297+0000", 2024-09-22T01:01:03.531 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3015865259999995, 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050790226000000001, 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2138 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198297+0000", 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198297+0000", 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198298+0000", 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198301+0000", 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1839999999999999e-06 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198310+0000", 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3060000000000003e-06 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198317+0000", 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7499999999999997e-06 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248837+0000", 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050520355000000003 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.248925+0000", 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7391000000000004e-05 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.249087+0000", 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016254999999999999 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2139 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3137536~5961] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198326+0000", 2024-09-22T01:01:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3015576220000007, 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051635923, 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2139 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198326+0000", 2024-09-22T01:01:03.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198326+0000", 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198327+0000", 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198344+0000", 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7266e-05 2024-09-22T01:01:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198353+0000", 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4800000000000007e-06 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198360+0000", 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8600000000000004e-06 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.249167+0000", 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050806401000000001 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.249350+0000", 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000182879 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.249962+0000", 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061242700000000002 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2140 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3227648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198370+0000", 2024-09-22T01:01:03.539 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3015140489999997, 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051957135000000002, 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2140 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198370+0000", 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198370+0000", 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198370+0000", 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198373+0000", 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0230000000000001e-06 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198382+0000", 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3240000000000003e-06 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198388+0000", 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.849e-06 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250065+0000", 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051676931000000002 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250173+0000", 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107509 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250327+0000", 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015385900000000001 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2141 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198408+0000", 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3014757899999996, 2024-09-22T01:01:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052463599, 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2141 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198408+0000", 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198408+0000", 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198409+0000", 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.540000000000001e-07 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198419+0000", 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9960000000000005e-06 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198428+0000", 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198435+0000", 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250415+0000", 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051980356999999998 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250607+0000", 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019222099999999999 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250871+0000", 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026423799999999997 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2142 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.547 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198447+0000", 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3014364230000002, 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052716038, 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2142 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198447+0000", 2024-09-22T01:01:03.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198447+0000", 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198448+0000", 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5000000000000001e-07 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198451+0000", 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198460+0000", 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6440000000000006e-06 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198477+0000", 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.250934+0000", 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052457384000000003 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251026+0000", 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1852000000000001e-05 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251163+0000", 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013750200000000001 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2143 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3497984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198488+0000", 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3013957729999994, 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053219743, 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2143 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198488+0000", 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198488+0000", 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198489+0000", 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198492+0000", 2024-09-22T01:01:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198502+0000", 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0530000000000001e-05 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198509+0000", 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6630000000000004e-06 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251246+0000", 2024-09-22T01:01:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052737447999999999 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251436+0000", 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018967300000000001 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251708+0000", 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027182600000000002 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2144 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3588096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198518+0000", 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3013657930000004, 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053478571000000003, 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2144 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198518+0000", 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198518+0000", 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198519+0000", 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198521+0000", 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8159999999999998e-06 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198530+0000", 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2600000000000005e-06 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198536+0000", 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8000000000000001e-06 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251768+0000", 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053231906000000002 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251860+0000", 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1848000000000006e-05 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.251996+0000", 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000136298 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2145 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198545+0000", 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3013385839999998, 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053976072999999999, 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2145 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198545+0000", 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198545+0000", 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198546+0000", 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198558+0000", 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2133000000000001e-05 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198567+0000", 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198574+0000", 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0829999999999998e-06 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252081+0000", 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053506747 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252263+0000", 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018252200000000001 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252521+0000", 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025806800000000002 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2146 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198612+0000", 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3012716809999993, 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054205883000000003, 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2146 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198612+0000", 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198612+0000", 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198613+0000", 2024-09-22T01:01:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.113e-06 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198619+0000", 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.677e-06 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198647+0000", 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8147e-05 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198657+0000", 2024-09-22T01:01:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9289999999999993e-06 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252582+0000", 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.053924723000000001 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252679+0000", 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7250999999999994e-05 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252818+0000", 2024-09-22T01:01:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013904299999999999 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2147 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198669+0000", 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3012151050000007, 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054677598000000001, 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2147 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198669+0000", 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198669+0000", 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198669+0000", 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0699999999999996e-07 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198673+0000", 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4829999999999999e-06 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198682+0000", 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4329999999999993e-06 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198689+0000", 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8970000000000002e-06 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.252900+0000", 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054210513000000002 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253085+0000", 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018475100000000001 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253346+0000", 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026161400000000002 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2148 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198700+0000", 2024-09-22T01:01:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3011839490000003, 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054943604, 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2148 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198700+0000", 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198700+0000", 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198700+0000", 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198703+0000", 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.813e-06 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198712+0000", 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4300000000000006e-06 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198718+0000", 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7399999999999998e-06 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253407+0000", 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054688771999999997 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253495+0000", 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8108000000000004e-05 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253643+0000", 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000148114 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2149 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 4038656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198727+0000", 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3011566220000006, 2024-09-22T01:01:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055436813000000001, 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2149 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198727+0000", 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198727+0000", 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198728+0000", 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198730+0000", 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6699999999999998e-06 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198739+0000", 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3499999999999997e-06 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198749+0000", 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.008e-05 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253726+0000", 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.054977005000000002 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.253911+0000", 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000185451 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.254164+0000", 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025263 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2150 3.1c 3:3a764c14:::rbd_data.117cf538ee01.0000000000000018:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.198757+0000", 2024-09-22T01:01:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3011263789999994, 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055676784, 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2150 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198757+0000", 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198757+0000", 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198758+0000", 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198772+0000", 2024-09-22T01:01:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4229999999999999e-05 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198789+0000", 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7187e-05 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.198796+0000", 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8360000000000003e-06 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.254208+0000", 2024-09-22T01:01:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055411929999999998 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.254299+0000", 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0420999999999994e-05 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.254434+0000", 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013554999999999999 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:03.585 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:03.585 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:03.586 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:03.586 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:03.584+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:03.687 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:03.684+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:03.715 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:03.715 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:03.729 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:03.788 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:03.784+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:03.807 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:01:03.855 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:03.855 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:03.855 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:03.855 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:03.855 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:03.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:03.889 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:03.888+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:03.962 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone5 from 0x6646ba1 (overlap 0x4adac7) to 0x0 2024-09-22T01:01:03.989 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:03.988+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:04.003 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:04.003 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2955 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6257336690000006, 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044281583999999999, 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2955 2024-09-22T01:01:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378128+0000", 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378131+0000", 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.864e-06 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378137+0000", 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8020000000000003e-06 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378141+0000", 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422313+0000", 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044172553000000003 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422359+0000", 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6146000000000002e-05 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422409+0000", 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9576000000000001e-05 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2956 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3284992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6257039810000009, 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044364010000000002, 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2956 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378159+0000", 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8679999999999999e-06 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378164+0000", 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2499999999999997e-06 2024-09-22T01:01:04.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378169+0000", 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3900000000000003e-06 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422438+0000", 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044268764000000002 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422478+0000", 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.066e-05 2024-09-22T01:01:04.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422521+0000", 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2741999999999998e-05 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2957 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6256872599999994, 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044474276, 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.012 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2957 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1299999999999999e-07 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378175+0000", 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3039999999999999e-06 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378179+0000", 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.388e-06 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378188+0000", 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0569999999999999e-06 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422550+0000", 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044362523000000001 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422597+0000", 2024-09-22T01:01:04.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6493999999999999e-05 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422648+0000", 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1297e-05 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2958 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6256672139999999, 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044568617999999997, 2024-09-22T01:01:04.016 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2958 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.75e-07 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378196+0000", 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4959999999999999e-06 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378199+0000", 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7969999999999998e-06 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378202+0000", 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.65e-06 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422678+0000", 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044475564000000002 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422718+0000", 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0825000000000003e-05 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422762+0000", 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3810999999999998e-05 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2959 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:04.020 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6256549570000001, 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044681193000000001, 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2959 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:04.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378207+0000", 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.316e-06 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378211+0000", 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378213+0000", 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.588e-06 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422794+0000", 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044580928999999998 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422840+0000", 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5231000000000001e-05 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422887+0000", 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7543999999999997e-05 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2960 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6256439119999992, 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044780915999999997, 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2960 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.025 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:04.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378218+0000", 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.032e-06 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378222+0000", 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4690000000000002e-06 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378224+0000", 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5040000000000001e-06 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422915+0000", 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044691036000000003 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422956+0000", 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0881999999999999e-05 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422998+0000", 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1782000000000001e-05 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2961 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6256332909999998, 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891911, 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2961 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378229+0000", 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.071e-06 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378232+0000", 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378235+0000", 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4609999999999998e-06 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423029+0000", 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044794498000000002 2024-09-22T01:01:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423072+0000", 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2796000000000002e-05 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423120+0000", 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7574999999999998e-05 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.033 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2962 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6256227600000006, 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044991058, 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2962 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378239+0000", 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378243+0000", 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3459999999999998e-06 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378245+0000", 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6340000000000001e-06 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423149+0000", 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044903169999999999 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423188+0000", 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8974999999999999e-05 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423229+0000", 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1655e-05 2024-09-22T01:01:04.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2963 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6256121480000001, 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104404000000001, 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.038 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378250+0000", 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.001e-06 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378255+0000", 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7949999999999998e-06 2024-09-22T01:01:04.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378264+0000", 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6129999999999993e-06 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423259+0000", 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044994348000000003 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423302+0000", 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3733000000000003e-05 2024-09-22T01:01:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423353+0000", 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0689000000000002e-05 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2964 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6255918600000001, 2024-09-22T01:01:04.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045200033000000001, 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2964 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378275+0000", 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3850000000000001e-06 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378279+0000", 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378281+0000", 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5129999999999999e-06 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423382+0000", 2024-09-22T01:01:04.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045100385 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423422+0000", 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0265999999999997e-05 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423469+0000", 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7110999999999997e-05 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.046 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2965 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6255693519999994, 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045300746000000003, 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2965 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6899999999999999e-07 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378294+0000", 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.951e-06 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378298+0000", 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1239999999999998e-06 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378301+0000", 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.779e-06 2024-09-22T01:01:04.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423497+0000", 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045196473000000001 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423544+0000", 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6622000000000002e-05 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423592+0000", 2024-09-22T01:01:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8528e-05 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2966 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6255559539999993, 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045402558000000003, 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2966 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378306+0000", 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2869999999999999e-06 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378310+0000", 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.258e-06 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378312+0000", 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7810000000000001e-06 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423622+0000", 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045309464000000001 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423663+0000", 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1310999999999998e-05 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423708+0000", 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4248000000000001e-05 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3013 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6255448569999995, 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045530159000000001, 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3013 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378317+0000", 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378321+0000", 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3110000000000001e-06 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378323+0000", 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5600000000000001e-06 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423737+0000", 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045414035999999998 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423781+0000", 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3279999999999999e-05 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423846+0000", 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5634000000000002e-05 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3014 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6255341990000005, 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045653157, 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3014 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378715+0000", 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038801599999999998 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378719+0000", 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4039999999999997e-06 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378731+0000", 2024-09-22T01:01:04.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1755e-05 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378739+0000", 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.712e-06 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423876+0000", 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045137505000000001 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423918+0000", 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2339999999999998e-05 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423980+0000", 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1425000000000002e-05 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3015 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.063 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6251068249999996, 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045369596999999998, 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3015 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378755+0000", 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0100000000000003e-07 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378759+0000", 2024-09-22T01:01:04.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378766+0000", 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.029e-06 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378769+0000", 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7189999999999999e-06 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424013+0000", 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045243798000000002 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424059+0000", 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5725e-05 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424124+0000", 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4851999999999995e-05 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3016 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6250854260000001, 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045478877000000001, 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3016 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379160+0000", 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038423800000000002 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379164+0000", 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379173+0000", 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.439e-06 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379178+0000", 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3909999999999997e-06 2024-09-22T01:01:04.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424155+0000", 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044976781 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424198+0000", 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769000000000003e-05 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424254+0000", 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6641000000000002e-05 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3017 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6246683869999998, 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045202985000000001, 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3017 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9400000000000005e-07 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379198+0000", 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379215+0000", 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6863000000000001e-05 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379219+0000", 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0749999999999999e-06 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424286+0000", 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045067013000000003 2024-09-22T01:01:04.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424332+0000", 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5531000000000002e-05 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424395+0000", 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3740999999999998e-05 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3018 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6246326339999992, 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045293472000000001, 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3018 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379430+0000", 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020146699999999999 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379435+0000", 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379443+0000", 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4819999999999994e-06 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379461+0000", 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.753e-05 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424424+0000", 2024-09-22T01:01:04.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044962716999999999 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424464+0000", 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0824000000000001e-05 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424522+0000", 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7383999999999998e-05 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3019 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6243903779999993, 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045194026999999998, 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3019 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381690+0000", 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022191849999999998 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381696+0000", 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3860000000000004e-06 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381705+0000", 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2480000000000002e-06 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381712+0000", 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.579e-06 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424554+0000", 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042842327999999999 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424601+0000", 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6697999999999999e-05 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424665+0000", 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3602999999999994e-05 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:8990 3.a 3:51e071dd:::rbd_data.117cc87baa96.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8228444790000005, 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048973669999999997, 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 8990 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181019+0000", 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4889999999999998e-06 2024-09-22T01:01:04.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181038+0000", 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9097e-05 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181066+0000", 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7514000000000001e-05 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181078+0000", 2024-09-22T01:01:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2626000000000001e-05 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181196+0000", 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118208 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181388+0000", 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019203100000000001 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.188674+0000", 2024-09-22T01:01:04.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072856120000000003 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229668+0000", 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040994282 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229764+0000", 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6036e-05 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229859+0000", 2024-09-22T01:01:04.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5088999999999997e-05 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229990+0000", 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130686 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.090 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:04.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:04.092 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:04.088+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:04.104 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-22T01:01:04.106 INFO:teuthology.orchestra.run.smithi002.stdout:726 clone 7 order 25 su 4194304 sc 7 2024-09-22T01:01:04.192 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:04.188+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:04.219 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:04.219 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:04.219 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:04.220 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:04.235 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:04.292 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:04.289+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:04.356 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:04.356 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:04.356 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:04.356 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:04.356 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:04.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:04.393 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:04.392+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:04.493 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:04.492+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:04.543 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:04.543 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:04.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:04.543 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:04.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.650643074000001, 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:01:04.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1907 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094113+0000", 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4477446559999994, 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056039765999999998, 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1907 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094113+0000", 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094113+0000", 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094115+0000", 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.457e-06 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094122+0000", 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094141+0000", 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.908e-05 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094155+0000", 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4276e-05 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.149936+0000", 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055781612000000001 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150045+0000", 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108498 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150153+0000", 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108087 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1908 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1671168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094173+0000", 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4476846139999999, 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056552275999999999, 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1908 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094173+0000", 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094173+0000", 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094175+0000", 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.24e-06 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094180+0000", 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1370000000000001e-06 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094197+0000", 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.717e-05 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094209+0000", 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2106999999999999e-05 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150243+0000", 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056034349999999997 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150464+0000", 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022080400000000001 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150726+0000", 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000261468 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1909 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094226+0000", 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4476315740000008, 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056756131000000001, 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.558 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1909 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094226+0000", 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094226+0000", 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094227+0000", 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.037e-06 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094233+0000", 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9830000000000001e-06 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094251+0000", 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7367000000000002e-05 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094266+0000", 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4763000000000001e-05 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150808+0000", 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056542800999999997 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150891+0000", 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2942999999999995e-05 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150983+0000", 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1236999999999994e-05 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1910 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094284+0000", 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4475739609999998, 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056887912999999998, 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1910 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094284+0000", 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094284+0000", 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094285+0000", 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094291+0000", 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.643e-06 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094308+0000", 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.766e-05 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094341+0000", 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2193000000000002e-05 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151027+0000", 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056685985000000001 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151097+0000", 2024-09-22T01:01:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0366999999999999e-05 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151172+0000", 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4998000000000003e-05 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1911 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1941504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094388+0000", 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4474694750000001, 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056956143000000001, 2024-09-22T01:01:04.566 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1911 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094388+0000", 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094388+0000", 2024-09-22T01:01:04.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094390+0000", 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1769999999999999e-06 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094397+0000", 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094418+0000", 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0659999999999999e-05 2024-09-22T01:01:04.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094433+0000", 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4946000000000001e-05 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151212+0000", 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056779863999999999 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151266+0000", 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3118000000000002e-05 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151345+0000", 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9005000000000007e-05 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1912 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094453+0000", 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4474053219999998, 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.057051945, 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.570 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1912 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094453+0000", 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094453+0000", 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094701+0000", 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000248454 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094710+0000", 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5099999999999998e-06 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094732+0000", 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2557000000000001e-05 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094749+0000", 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6739999999999999e-05 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151381+0000", 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056631853000000003 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151434+0000", 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2967000000000003e-05 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151505+0000", 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0864000000000005e-05 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1913 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2121728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094773+0000", 2024-09-22T01:01:04.574 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4470853310000003, 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056874896000000001, 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1913 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094773+0000", 2024-09-22T01:01:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094773+0000", 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094774+0000", 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3859999999999999e-06 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094798+0000", 2024-09-22T01:01:04.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4326999999999999e-05 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094819+0000", 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0573000000000001e-05 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094834+0000", 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4966e-05 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151537+0000", 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056702874 2024-09-22T01:01:04.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151582+0000", 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4827000000000003e-05 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151648+0000", 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5943000000000002e-05 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1914 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094853+0000", 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4470046889999999, 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056940442000000001, 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1914 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094853+0000", 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094853+0000", 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096093+0000", 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012396900000000001 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096102+0000", 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1570000000000006e-06 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096129+0000", 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6622999999999999e-05 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096147+0000", 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8114e-05 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151680+0000", 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055532893 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151726+0000", 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6233000000000003e-05 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151794+0000", 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7731999999999995e-05 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1918 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2572288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096474+0000", 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4453844409999999, 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056054800000000002, 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1918 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096474+0000", 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096474+0000", 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096475+0000", 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-09-22T01:01:04.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096481+0000", 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096499+0000", 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8332999999999999e-05 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096512+0000", 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.346e-05 2024-09-22T01:01:04.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152430+0000", 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055917426999999999 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152478+0000", 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7988999999999997e-05 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152528+0000", 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0504999999999998e-05 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1919 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2662400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096530+0000", 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4453284419999992, 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056125652999999998, 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1919 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096530+0000", 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096530+0000", 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096531+0000", 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096536+0000", 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096554+0000", 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.732e-05 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096584+0000", 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0762999999999997e-05 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152561+0000", 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055976763999999998 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152607+0000", 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5405999999999999e-05 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152655+0000", 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8622999999999998e-05 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1920 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096603+0000", 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4452549290000007, 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056177865, 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.591 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1920 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096603+0000", 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096603+0000", 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096604+0000", 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096610+0000", 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096628+0000", 2024-09-22T01:01:04.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8369999999999999e-05 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096642+0000", 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.415e-05 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152687+0000", 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056044683999999997 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152732+0000", 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4613999999999997e-05 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152781+0000", 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9296999999999999e-05 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1921 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2842624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096659+0000", 2024-09-22T01:01:04.595 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4451986990000005, 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056245989000000003, 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1921 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096659+0000", 2024-09-22T01:01:04.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096659+0000", 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096661+0000", 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.246e-06 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096666+0000", 2024-09-22T01:01:04.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2340000000000002e-06 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096683+0000", 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7493e-05 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096697+0000", 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3859999999999999e-05 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152813+0000", 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056116054999999998 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152858+0000", 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4690000000000001e-05 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152905+0000", 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7410999999999997e-05 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1922 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2932736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096714+0000", 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4451437269999996, 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056318841000000001, 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1922 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096714+0000", 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096714+0000", 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096715+0000", 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0270000000000001e-06 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096720+0000", 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096738+0000", 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7105999999999999e-05 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096751+0000", 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3617e-05 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152936+0000", 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056185156999999999 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152984+0000", 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.774e-05 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153033+0000", 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9044e-05 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.603 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1923 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3022848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096768+0000", 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4450895040000002, 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056392314999999998, 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1923 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096768+0000", 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096768+0000", 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096770+0000", 2024-09-22T01:01:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.097e-06 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096775+0000", 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1070000000000004e-06 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096792+0000", 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7706e-05 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096806+0000", 2024-09-22T01:01:04.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153066+0000", 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056260403 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153113+0000", 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6464000000000001e-05 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153161+0000", 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7994000000000002e-05 2024-09-22T01:01:04.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1924 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096823+0000", 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4450351000000001, 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056462601000000001, 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1924 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096823+0000", 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096823+0000", 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096824+0000", 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.003e-06 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096845+0000", 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1376e-05 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096864+0000", 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8810000000000001e-05 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096878+0000", 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4136e-05 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153192+0000", 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056314125 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153237+0000", 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4721000000000001e-05 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153285+0000", 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8430000000000002e-05 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1925 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3203072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096897+0000", 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4449614480000008, 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056516714000000003, 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.612 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1925 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096897+0000", 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096897+0000", 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096898+0000", 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096904+0000", 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6069999999999998e-06 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096921+0000", 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7799999999999999e-05 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096935+0000", 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3983e-05 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153317+0000", 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056381477999999999 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153365+0000", 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:01:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153413+0000", 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7839000000000001e-05 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1926 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096953+0000", 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.444905082, 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056584061999999997, 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.616 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1926 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096953+0000", 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096953+0000", 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097198+0000", 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024509500000000002 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097207+0000", 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0939999999999997e-06 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097229+0000", 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2436e-05 2024-09-22T01:01:04.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097246+0000", 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6603e-05 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153444+0000", 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056197902000000001 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153489+0000", 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4706000000000002e-05 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153537+0000", 2024-09-22T01:01:04.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8226000000000002e-05 2024-09-22T01:01:04.620 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.620 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.620 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.620 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1927 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3383296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.097275+0000", 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4445827680000001, 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056399983000000001, 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1927 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.621 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097275+0000", 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097275+0000", 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097875+0000", 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059935100000000003 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097893+0000", 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8386e-05 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097918+0000", 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4674000000000001e-05 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097953+0000", 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5311000000000001e-05 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153569+0000", 2024-09-22T01:01:04.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055615708 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153614+0000", 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4966999999999998e-05 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153675+0000", 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1586000000000003e-05 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.625 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1931 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.098137+0000", 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4437211019999996, 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056042361999999998, 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:04.626 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1931 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.098137+0000", 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.098137+0000", 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.101915+0000", 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003778563 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.101927+0000", 2024-09-22T01:01:04.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1576999999999999e-05 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.101954+0000", 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7073999999999999e-05 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.101973+0000", 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8893999999999999e-05 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.154087+0000", 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052113605 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.154132+0000", 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5061000000000001e-05 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.154179+0000", 2024-09-22T01:01:04.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7587999999999999e-05 2024-09-22T01:01:04.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.631 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:04.631 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:04.631 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:04.632 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:04.633 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:04.628+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:04.733 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:04.729+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:04.767 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:04.767 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:04.767 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:04.767 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:04.783 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:04.812 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:01:04.834 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:04.832+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:04.919 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:04.919 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:04.919 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:04.919 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:04.919 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:04.935 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:04.935 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:04.932+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:04.967 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone6 intact 2024-09-22T01:01:05.006 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:05.057 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:05.056+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8692 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0234680550000004, 2024-09-22T01:01:05.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01097323, 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8692 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089896+0000", 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089904+0000", 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3440000000000001e-06 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089911+0000", 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5799999999999997e-06 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100463+0000", 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010552713 2024-09-22T01:01:05.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100732+0000", 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026876599999999998 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100866+0000", 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133623 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.119 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8693 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0234410150000004, 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011283726000000001, 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8693 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089924+0000", 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089932+0000", 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9929999999999995e-06 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089941+0000", 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9719999999999998e-06 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100989+0000", 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011048231 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101092+0000", 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102975 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101203+0000", 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111648 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8712 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 630784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.023340052, 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011403979999999999, 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8712 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090022+0000", 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.19e-06 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090027+0000", 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.604e-06 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090041+0000", 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3114e-05 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090049+0000", 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6479999999999992e-06 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101235+0000", 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011185428000000001 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101309+0000", 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4376999999999996e-05 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101425+0000", 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011561900000000001 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8713 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0232957960000002, 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011573441, 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8713 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090069+0000", 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9840000000000003e-06 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090077+0000", 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6639999999999998e-06 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090083+0000", 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101474+0000", 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011391007 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101548+0000", 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4555999999999996e-05 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101638+0000", 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0223000000000007e-05 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8737 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0232701669999997, 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011746902, 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8737 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:05.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090093+0000", 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1619999999999998e-06 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090099+0000", 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2249999999999997e-06 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090104+0000", 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9520000000000002e-06 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101668+0000", 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011563339000000001 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101743+0000", 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5796000000000004e-05 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101837+0000", 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4055999999999995e-05 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8747 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 729088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:05.136 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0232476449999997, 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011923398, 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8747 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9900000000000001e-07 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090127+0000", 2024-09-22T01:01:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3447e-05 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090134+0000", 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3799999999999996e-06 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090139+0000", 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8119999999999998e-06 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101866+0000", 2024-09-22T01:01:05.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011726752 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101938+0000", 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102036+0000", 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8041000000000005e-05 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.140 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8757 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0232126770000001, 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012330969000000001, 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8757 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:01:05.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090151+0000", 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090159+0000", 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4869999999999996e-06 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090164+0000", 2024-09-22T01:01:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3210000000000004e-06 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102065+0000", 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011901022000000001 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102381+0000", 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031597199999999998 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102479+0000", 2024-09-22T01:01:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8101000000000001e-05 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8767 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0231890290000001, 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012517782, 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8767 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090174+0000", 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090181+0000", 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6320000000000002e-06 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090186+0000", 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6949999999999999e-06 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102511+0000", 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012325469 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102601+0000", 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9802000000000005e-05 2024-09-22T01:01:05.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102689+0000", 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8571000000000003e-05 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8777 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0231644009999998, 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012693987, 2024-09-22T01:01:05.149 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8777 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090197+0000", 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090199+0000", 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090206+0000", 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4089999999999997e-06 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090212+0000", 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4380000000000003e-06 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102718+0000", 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012506735999999999 2024-09-22T01:01:05.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102793+0000", 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5054000000000003e-05 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102890+0000", 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6825999999999998e-05 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.153 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8787 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 786432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0218855600000003, 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011612116, 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8787 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091476+0000", 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.11e-06 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091482+0000", 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091494+0000", 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2047e-05 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091503+0000", 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102918+0000", 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011414951 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102992+0000", 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3279000000000001e-05 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103087+0000", 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5662000000000007e-05 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.157 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8797 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 802816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0218414029999998, 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011753412, 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8797 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091520+0000", 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5000000000000003e-07 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091536+0000", 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6214000000000001e-05 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091546+0000", 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7799999999999995e-06 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091559+0000", 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2911000000000001e-05 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103116+0000", 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011557207 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103188+0000", 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2418999999999999e-05 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103273+0000", 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4331000000000003e-05 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8807 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 811008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0217890389999997, 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011888717, 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8807 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091576+0000", 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5540000000000002e-06 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091586+0000", 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0502999999999999e-05 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091593+0000", 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.861e-06 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103300+0000", 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011706467999999999 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103371+0000", 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1308000000000002e-05 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103460+0000", 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9395999999999996e-05 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8818 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 761856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0217571269999999, 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012045223000000001, 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8818 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2200000000000004e-07 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091607+0000", 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5440000000000001e-06 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091615+0000", 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0169999999999996e-06 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091620+0000", 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-06 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103488+0000", 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011867353000000001 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103561+0000", 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3481999999999999e-05 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103649+0000", 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7495000000000002e-05 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8821 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.021731054, 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012214868, 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8821 2024-09-22T01:01:05.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3099999999999998e-07 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091633+0000", 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5770000000000001e-06 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091641+0000", 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091647+0000", 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1079999999999998e-06 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103676+0000", 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012029361000000001 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103747+0000", 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0587999999999996e-05 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103844+0000", 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7606e-05 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8822 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0217047519999998, 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01239211, 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8822 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091659+0000", 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5770000000000001e-06 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091667+0000", 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3410000000000003e-06 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091673+0000", 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8270000000000003e-06 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103872+0000", 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012199162 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103951+0000", 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8512999999999999e-05 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104048+0000", 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7212000000000004e-05 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8824 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0216789950000003, 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012562043, 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8824 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091685+0000", 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5349999999999999e-06 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091693+0000", 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3019999999999995e-06 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091707+0000", 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4345000000000001e-05 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104080+0000", 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012372321 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104156+0000", 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6734000000000001e-05 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104244+0000", 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7363999999999999e-05 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8848 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0203133610000004, 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011398933, 2024-09-22T01:01:05.183 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8848 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093048+0000", 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.136e-06 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093054+0000", 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6969999999999998e-06 2024-09-22T01:01:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093074+0000", 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.95e-05 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093083+0000", 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3479999999999993e-06 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104273+0000", 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011189747999999999 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104352+0000", 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9560000000000004e-05 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104446+0000", 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3943999999999994e-05 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8849 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 892928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0202610070000002, 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011546511000000001, 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8849 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093104+0000", 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4010000000000001e-06 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093113+0000", 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1349999999999998e-06 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093119+0000", 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3720000000000001e-06 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104477+0000", 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011358157000000001 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104554+0000", 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6390000000000006e-05 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104646+0000", 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2379000000000004e-05 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8868 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 860160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0202308599999999, 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011726966, 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8868 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5999999999999999e-07 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093133+0000", 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6900000000000001e-06 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093142+0000", 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6780000000000006e-06 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093148+0000", 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1160000000000004e-06 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104677+0000", 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011529285 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104756+0000", 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104857+0000", 2024-09-22T01:01:05.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010123099999999999 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8869 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0201975579999996, 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011914377, 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:05.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8869 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:05.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093164+0000", 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093167+0000", 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4359999999999998e-06 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093176+0000", 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9590000000000001e-06 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:05.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093193+0000", 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6986999999999998e-05 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104890+0000", 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011696745 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104969+0000", 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9086999999999997e-05 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.105078+0000", 2024-09-22T01:01:05.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108503 2024-09-22T01:01:05.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:05.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:05.200 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:05.201 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:05.202 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:05.202 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:05.202 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:05.203 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:05.203 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-22T01:01:05.203 INFO:teuthology.orchestra.run.smithi002.stdout:727 trunc from 0x6646ba1 to 0x35ea6b6 2024-09-22T01:01:05.203 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:05.200+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:05.204 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:05.205 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:05.238 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:05.238 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:05.261 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:05.261 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:05.261 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:05.287 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:05.302 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:05.300+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:05.310 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:05.311 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:05.314 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:05.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:05.337 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:05.338 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:05.356 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:05.371 INFO:teuthology.orchestra.run.smithi002.stdout:728 writesame 0xd0b2e23 thru 0xd0bb015 data_size 0xd(0x81f3 bytes) 2024-09-22T01:01:05.373 INFO:teuthology.orchestra.run.smithi002.stdout:729 punch from 0x3bf7433 to 0x3c06d31, (0xf8fe bytes) 2024-09-22T01:01:05.373 INFO:teuthology.orchestra.run.smithi002.stdout:730 read 0x928933a thru 0x9289ef2 (0xbb9 bytes) 2024-09-22T01:01:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:731 write 0xbdbebdc thru 0xbdc645e (0x7883 bytes) 2024-09-22T01:01:05.376 INFO:teuthology.orchestra.run.smithi002.stdout:732 write 0x2c60d0c thru 0x2c6206f (0x1364 bytes) 2024-09-22T01:01:05.379 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:05.379 INFO:teuthology.orchestra.run.smithi002.stdout:734 punch from 0x80fac21 to 0x80fc2c6, (0x16a5 bytes) 2024-09-22T01:01:05.379 INFO:teuthology.orchestra.run.smithi002.stdout:735 read 0x2217ba3 thru 0x22247a0 (0xcbfe bytes) 2024-09-22T01:01:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:737 trunc from 0xd0bb016 to 0x2006045 2024-09-22T01:01:05.403 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:05.397+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:05.412 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:05.427 INFO:teuthology.orchestra.run.smithi002.stdout:742 trunc from 0x2006045 to 0xbdb8499 2024-09-22T01:01:05.436 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:05.436 INFO:teuthology.orchestra.run.smithi002.stdout:744 read 0x8ccfaf8 thru 0x8cd6c4e (0x7157 bytes) 2024-09-22T01:01:05.437 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:05.437 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:05.437 INFO:teuthology.orchestra.run.smithi002.stdout:750 write 0xb31cf92 thru 0xb325dc6 (0x8e35 bytes) 2024-09-22T01:01:05.439 INFO:teuthology.orchestra.run.smithi002.stdout:752 write 0xfacfad thru 0xfbbf05 (0xef59 bytes) 2024-09-22T01:01:05.442 INFO:teuthology.orchestra.run.smithi002.stdout:753 read 0x3a34172 thru 0x3a41822 (0xd6b1 bytes) 2024-09-22T01:01:05.443 INFO:teuthology.orchestra.run.smithi002.stdout:755 read 0x178b3f3 thru 0x179130d (0x5f1b bytes) 2024-09-22T01:01:05.443 INFO:teuthology.orchestra.run.smithi002.stdout:756 write 0x6f37ba2 thru 0x6f4727c (0xf6db bytes) 2024-09-22T01:01:05.447 INFO:teuthology.orchestra.run.smithi002.stdout:757 read 0x7541a33 thru 0x754bbc0 (0xa18e bytes) 2024-09-22T01:01:05.447 INFO:teuthology.orchestra.run.smithi002.stdout:758 read 0x24208e8 thru 0x242f0c1 (0xe7da bytes) 2024-09-22T01:01:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:761 read 0x71be388 thru 0x71c35fc (0x5275 bytes) 2024-09-22T01:01:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:762 writesame 0x8695d92 thru 0x869e98c data_size 0x5(0x8bfb bytes) 2024-09-22T01:01:05.453 INFO:teuthology.orchestra.run.smithi002.stdout:763 trunc from 0xbdb8499 to 0xba6d80a 2024-09-22T01:01:05.477 INFO:teuthology.orchestra.run.smithi002.stdout:764 trunc from 0xba6d80a to 0x34eb82c 2024-09-22T01:01:05.491 INFO:teuthology.orchestra.run.smithi002.stdout:765 write 0x17c2497 thru 0x17c7b47 (0x56b1 bytes) 2024-09-22T01:01:05.493 INFO:teuthology.orchestra.run.smithi002.stdout:766 read 0x239d159 thru 0x23ab1c5 (0xe06d bytes) 2024-09-22T01:01:05.493 INFO:teuthology.orchestra.run.smithi002.stdout:767 read 0xc6f3db thru 0xc7d3c4 (0xdfea bytes) 2024-09-22T01:01:05.495 INFO:teuthology.orchestra.run.smithi002.stdout:768 punch from 0x10b09cd to 0x10be395, (0xd9c8 bytes) 2024-09-22T01:01:05.495 INFO:teuthology.orchestra.run.smithi002.stdout:769 write 0x28fd5dc thru 0x28fdbbb (0x5e0 bytes) 2024-09-22T01:01:05.498 INFO:teuthology.orchestra.run.smithi002.stdout:770 read 0x98eab7 thru 0x98fd47 (0x1291 bytes) 2024-09-22T01:01:05.504 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:05.500+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:05.506 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:05.507 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:05.511 INFO:teuthology.orchestra.run.smithi002.stdout:771 write 0x4e0e242 thru 0x4e17ff5 (0x9db4 bytes) 2024-09-22T01:01:05.526 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:05.533 INFO:teuthology.orchestra.run.smithi002.stdout:777 trunc from 0x4e17ff6 to 0xe8bab15 2024-09-22T01:01:05.542 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:05.543 INFO:teuthology.orchestra.run.smithi002.stdout:779 read 0x536c43 thru 0x53a682 (0x3a40 bytes) 2024-09-22T01:01:05.544 INFO:teuthology.orchestra.run.smithi002.stdout:781 trunc from 0xe8bab15 to 0xdae9294 2024-09-22T01:01:05.579 INFO:teuthology.orchestra.run.smithi002.stdout:782 trunc from 0xdae9294 to 0xc7953 2024-09-22T01:01:05.606 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:05.601+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:05.629 INFO:teuthology.orchestra.run.smithi002.stdout:783 clone 8 order 22 su 131072 sc 7 2024-09-22T01:01:05.669 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:05.669 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:05.683 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:05.725 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:05.724+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:05.807 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:05.804+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:05.863 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:05.863 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:05.879 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:05.908 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:05.905+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:06.009 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:06.005+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:06.035 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone7 from 0xc7953 (overlap 0xc7953) to 0x0 2024-09-22T01:01:06.058 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:06.058 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:06.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:06.121 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:06.120+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:06.210 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:06.205+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:06.215 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:06.215 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:06.232 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:06.310 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-22T01:01:06.311 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:06.308+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:06.411 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:06.409+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:06.430 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:06.430 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:06.446 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:06.513 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:06.513+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:06.614 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:06.612+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:06.622 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:06.622 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:06.628 INFO:teuthology.orchestra.run.smithi002.stdout:784 read 0x54d33 thru 0x5fa60 (0xad2e bytes) 2024-09-22T01:01:06.632 INFO:teuthology.orchestra.run.smithi002.stdout:787 punch from 0x48bfc to 0x51bbf, (0x8fc3 bytes) 2024-09-22T01:01:06.638 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:06.653 INFO:teuthology.orchestra.run.smithi002.stdout:789 writesame 0x55b8547 thru 0x55c1d94 data_size 0x23(0x984e bytes) 2024-09-22T01:01:06.653 INFO:teuthology.orchestra.run.smithi002.stdout:790 read 0xc3ed20 thru 0xc4c412 (0xd6f3 bytes) 2024-09-22T01:01:06.656 INFO:teuthology.orchestra.run.smithi002.stdout:791 trunc from 0x55c1d95 to 0x8327997 2024-09-22T01:01:06.664 INFO:teuthology.orchestra.run.smithi002.stdout:792 writesame 0x2a5b63e thru 0x2a68b17 data_size 0x5(0xd4da bytes) 2024-09-22T01:01:06.695 INFO:teuthology.orchestra.run.smithi002.stdout:793 writesame 0xdb24a16 thru 0xdb3445f data_size 0x76(0xfa4a bytes) 2024-09-22T01:01:06.720 INFO:teuthology.orchestra.run.smithi002.stdout:794 writesame 0x797d800 thru 0x797fa3e data_size 0x1(0x223f bytes) 2024-09-22T01:01:06.721 INFO:teuthology.orchestra.run.smithi002.stdout:795 writesame 0xaafe572 thru 0xab066c0 data_size 0x1(0x814f bytes) 2024-09-22T01:01:06.721 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:06.720+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:06.737 INFO:teuthology.orchestra.run.smithi002.stdout:796 punch from 0xe329c9 to 0xe37591, (0x4bc8 bytes) 2024-09-22T01:01:06.743 INFO:teuthology.orchestra.run.smithi002.stdout:797 trunc from 0xdb34460 to 0xa7d7387 2024-09-22T01:01:06.761 INFO:teuthology.orchestra.run.smithi002.stdout:798 trunc from 0xa7d7387 to 0xb3f5294 2024-09-22T01:01:06.771 INFO:teuthology.orchestra.run.smithi002.stdout:799 read 0x217063 thru 0x218031 (0xfcf bytes) 2024-09-22T01:01:06.771 INFO:teuthology.orchestra.run.smithi002.stdout:800 read 0x72e8735 thru 0x72ec20a (0x3ad6 bytes) 2024-09-22T01:01:06.787 INFO:teuthology.orchestra.run.smithi002.stdout:801 writesame 0xe6f967d thru 0xe700f82 data_size 0x7(0x7906 bytes) 2024-09-22T01:01:06.787 INFO:teuthology.orchestra.run.smithi002.stdout:805 write 0xd7efdda thru 0xd7f3e03 (0x402a bytes) 2024-09-22T01:01:06.792 INFO:teuthology.orchestra.run.smithi002.stdout:806 punch from 0x1dccff6 to 0x1dd1a4b, (0x4a55 bytes) 2024-09-22T01:01:06.801 INFO:teuthology.orchestra.run.smithi002.stdout:807 trunc from 0xe700f83 to 0x988ba65 2024-09-22T01:01:06.813 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:06.814 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:06.822 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:06.817+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:06.823 INFO:teuthology.orchestra.run.smithi002.stdout:809 read 0x7e960c2 thru 0x7e969df (0x91e bytes) 2024-09-22T01:01:06.823 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:06.823 INFO:teuthology.orchestra.run.smithi002.stdout:811 write 0x2bb685f thru 0x2bc1ee2 (0xb684 bytes) 2024-09-22T01:01:06.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:06.838 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:01:06.845 INFO:teuthology.orchestra.run.smithi002.stdout:812 trunc from 0x988ba65 to 0x23bd010 2024-09-22T01:01:06.862 INFO:teuthology.orchestra.run.smithi002.stdout:813 write 0x716a0b thru 0x71820e (0x1804 bytes) 2024-09-22T01:01:06.869 INFO:teuthology.orchestra.run.smithi002.stdout:814 write 0x7117c6 thru 0x71f89d (0xe0d8 bytes) 2024-09-22T01:01:06.872 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:06.886 INFO:teuthology.orchestra.run.smithi002.stdout:816 write 0x6025527 thru 0x60298da (0x43b4 bytes) 2024-09-22T01:01:06.890 INFO:teuthology.orchestra.run.smithi002.stdout:818 read 0x2a4b693 thru 0x2a5912a (0xda98 bytes) 2024-09-22T01:01:06.891 INFO:teuthology.orchestra.run.smithi002.stdout:821 writesame 0x3b475b6 thru 0x3b54642 data_size 0x1d(0xd08d bytes) 2024-09-22T01:01:06.896 INFO:teuthology.orchestra.run.smithi002.stdout:824 writesame 0x1893876 thru 0x18a27f8 data_size 0x1(0xef83 bytes) 2024-09-22T01:01:06.923 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:06.920+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:06.929 INFO:teuthology.orchestra.run.smithi002.stdout:825 read 0x20b28f8 thru 0x20bae3d (0x8546 bytes) 2024-09-22T01:01:06.930 INFO:teuthology.orchestra.run.smithi002.stdout:826 read 0x3b2b088 thru 0x3b31fdd (0x6f56 bytes) 2024-09-22T01:01:06.931 INFO:teuthology.orchestra.run.smithi002.stdout:827 trunc from 0x60298db to 0x51b0987 2024-09-22T01:01:06.959 INFO:teuthology.orchestra.run.smithi002.stdout:828 trunc from 0x51b0987 to 0xcb64417 2024-09-22T01:01:06.966 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:06.966 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:06.968 INFO:teuthology.orchestra.run.smithi002.stdout:829 punch from 0x92eebba to 0x92fcb21, (0xdf67 bytes) 2024-09-22T01:01:06.968 INFO:teuthology.orchestra.run.smithi002.stdout:831 read 0xaa8a2c1 thru 0xaa99d17 (0xfa57 bytes) 2024-09-22T01:01:06.968 INFO:teuthology.orchestra.run.smithi002.stdout:832 read 0x30659c2 thru 0x306e731 (0x8d70 bytes) 2024-09-22T01:01:06.969 INFO:teuthology.orchestra.run.smithi002.stdout:833 read 0xb4ad43c thru 0xb4b0cb1 (0x3876 bytes) 2024-09-22T01:01:06.969 INFO:teuthology.orchestra.run.smithi002.stdout:834 write 0x3ab9fdc thru 0x3ac1485 (0x74aa bytes) 2024-09-22T01:01:06.975 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:06.975 INFO:teuthology.orchestra.run.smithi002.stdout:836 writesame 0xa56dc4 thru 0xa5da04 data_size 0x25(0x6c41 bytes) 2024-09-22T01:01:06.982 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:07.002 INFO:teuthology.orchestra.run.smithi002.stdout:838 trunc from 0xcb64417 to 0x1e4b9ab 2024-09-22T01:01:07.020 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-22T01:01:07.020 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:01:07.020 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-22T01:01:07.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-22T01:01:07.027 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:07.025+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:07.034 INFO:teuthology.orchestra.run.smithi002.stdout:839 trunc from 0x1e4b9ab to 0xddca843 2024-09-22T01:01:07.057 INFO:teuthology.orchestra.run.smithi002.stdout:840 trunc from 0xddca843 to 0x1be59bd 2024-09-22T01:01:07.074 INFO:teuthology.orchestra.run.smithi002.stdout:843 punch from 0x7e8678 to 0x7f6115, (0xda9d bytes) 2024-09-22T01:01:07.086 INFO:teuthology.orchestra.run.smithi002.stdout:845 write 0x5c08d7e thru 0x5c0cc70 (0x3ef3 bytes) 2024-09-22T01:01:07.102 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:07.102 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:07.106 INFO:teuthology.orchestra.run.smithi002.stdout:847 writesame 0xc375212 thru 0xc381c16 data_size 0x1(0xca05 bytes) 2024-09-22T01:01:07.107 INFO:teuthology.orchestra.run.smithi002.stdout:848 writesame 0x11eb58e thru 0x11f8da1 data_size 0x2(0xd814 bytes) 2024-09-22T01:01:07.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:07.131 INFO:teuthology.orchestra.run.smithi002.stdout:849 trunc from 0xc381c17 to 0x84d683c 2024-09-22T01:01:07.138 INFO:teuthology.orchestra.run.smithi002.stdout:850 read 0x62f61fd thru 0x62fbb66 (0x596a bytes) 2024-09-22T01:01:07.140 INFO:teuthology.orchestra.run.smithi002.stdout:852 trunc from 0x84d683c to 0x9c3ac65 2024-09-22T01:01:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:853 read 0x60f0c08 thru 0x60f7b3f (0x6f38 bytes) 2024-09-22T01:01:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.152 INFO:teuthology.orchestra.run.smithi002.stdout:856 trunc from 0x9c3ac65 to 0xd142ec8 2024-09-22T01:01:07.160 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:07.157+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:07.160 INFO:teuthology.orchestra.run.smithi002.stdout:857 writesame 0xa4750a3 thru 0xa481018 data_size 0x3f(0xbf76 bytes) 2024-09-22T01:01:07.164 INFO:teuthology.orchestra.run.smithi002.stdout:858 punch from 0x2ef89d0 to 0x2eff9d4, (0x7004 bytes) 2024-09-22T01:01:07.164 INFO:teuthology.orchestra.run.smithi002.stdout:860 read 0x5124e8d thru 0x51347bb (0xf92f bytes) 2024-09-22T01:01:07.165 INFO:teuthology.orchestra.run.smithi002.stdout:861 writesame 0x479e993 thru 0x47a1bfb data_size 0x1d(0x3269 bytes) 2024-09-22T01:01:07.170 INFO:teuthology.orchestra.run.smithi002.stdout:863 read 0x3544d thru 0x43b8e (0xe742 bytes) 2024-09-22T01:01:07.171 INFO:teuthology.orchestra.run.smithi002.stdout:865 read 0x5693a26 thru 0x569a1f1 (0x67cc bytes) 2024-09-22T01:01:07.171 INFO:teuthology.orchestra.run.smithi002.stdout:866 read 0x17b41cd thru 0x17bd529 (0x935d bytes) 2024-09-22T01:01:07.172 INFO:teuthology.orchestra.run.smithi002.stdout:868 write 0x2ff2902 thru 0x2ffdd75 (0xb474 bytes) 2024-09-22T01:01:07.176 INFO:teuthology.orchestra.run.smithi002.stdout:870 read 0x8d387dc thru 0x8d45857 (0xd07c bytes) 2024-09-22T01:01:07.177 INFO:teuthology.orchestra.run.smithi002.stdout:871 write 0x80a654c thru 0x80ae21d (0x7cd2 bytes) 2024-09-22T01:01:07.179 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.179 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.179 INFO:teuthology.orchestra.run.smithi002.stdout:874 write 0xc6f784a thru 0xc6f92cc (0x1a83 bytes) 2024-09-22T01:01:07.183 INFO:teuthology.orchestra.run.smithi002.stdout:876 read 0xb7c4937 thru 0xb7ce88f (0x9f59 bytes) 2024-09-22T01:01:07.183 INFO:teuthology.orchestra.run.smithi002.stdout:878 read 0x9683d9c thru 0x968e7f0 (0xaa55 bytes) 2024-09-22T01:01:07.192 INFO:teuthology.orchestra.run.smithi002.stdout:879 writesame 0xdd15407 thru 0xdd16385 data_size 0x1(0xf7f bytes) 2024-09-22T01:01:07.192 INFO:teuthology.orchestra.run.smithi002.stdout:880 read 0xa6fc705 thru 0xa70047a (0x3d76 bytes) 2024-09-22T01:01:07.192 INFO:teuthology.orchestra.run.smithi002.stdout:881 read 0x27d4c1a thru 0x27dab0a (0x5ef1 bytes) 2024-09-22T01:01:07.193 INFO:teuthology.orchestra.run.smithi002.stdout:884 writesame 0x2be57ed thru 0x2bead0e data_size 0x11(0x5522 bytes) 2024-09-22T01:01:07.195 INFO:teuthology.orchestra.run.smithi002.stdout:885 read 0x7163705 thru 0x7171569 (0xde65 bytes) 2024-09-22T01:01:07.197 INFO:teuthology.orchestra.run.smithi002.stdout:886 trunc from 0xdd16386 to 0xcfcc6ab 2024-09-22T01:01:07.205 INFO:teuthology.orchestra.run.smithi002.stdout:887 read 0x5bf0583 thru 0x5bffe43 (0xf8c1 bytes) 2024-09-22T01:01:07.205 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.212 INFO:teuthology.orchestra.run.smithi002.stdout:890 write 0xd70153a thru 0xd707724 (0x61eb bytes) 2024-09-22T01:01:07.214 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-22T01:01:07.217 INFO:teuthology.orchestra.run.smithi002.stdout:891 punch from 0x9f97ab to 0xa0199f, (0x81f4 bytes) 2024-09-22T01:01:07.217 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.217 INFO:teuthology.orchestra.run.smithi002.stdout:893 punch from 0x627a3f9 to 0x6288227, (0xde2e bytes) 2024-09-22T01:01:07.217 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.217 INFO:teuthology.orchestra.run.smithi002.stdout:895 write 0x646f181 thru 0x646f7ae (0x62e bytes) 2024-09-22T01:01:07.221 INFO:teuthology.orchestra.run.smithi002.stdout:897 read 0x49f5f4c thru 0x49f8e6c (0x2f21 bytes) 2024-09-22T01:01:07.221 INFO:teuthology.orchestra.run.smithi002.stdout:898 read 0xcb4cb7a thru 0xcb5729c (0xa723 bytes) 2024-09-22T01:01:07.222 INFO:teuthology.orchestra.run.smithi002.stdout:900 punch from 0x21ce070 to 0x21dc53d, (0xe4cd bytes) 2024-09-22T01:01:07.222 INFO:teuthology.orchestra.run.smithi002.stdout:901 read 0x33b43be thru 0x33c28e2 (0xe525 bytes) 2024-09-22T01:01:07.222 INFO:teuthology.orchestra.run.smithi002.stdout:904 read 0x38fc009 thru 0x3906365 (0xa35d bytes) 2024-09-22T01:01:07.222 INFO:teuthology.orchestra.run.smithi002.stdout:905 writesame 0xb6b351a thru 0xb6b695b data_size 0x1(0x3442 bytes) 2024-09-22T01:01:07.225 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:07.221+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:07.229 INFO:teuthology.orchestra.run.smithi002.stdout:906 punch from 0x2203c97 to 0x220b0b2, (0x741b bytes) 2024-09-22T01:01:07.229 INFO:teuthology.orchestra.run.smithi002.stdout:907 write 0x4160cfc thru 0x41618e0 (0xbe5 bytes) 2024-09-22T01:01:07.232 INFO:teuthology.orchestra.run.smithi002.stdout:908 read 0x1213f40 thru 0x12223bc (0xe47d bytes) 2024-09-22T01:01:07.241 INFO:teuthology.orchestra.run.smithi002.stdout:910 trunc from 0xd707725 to 0x7b3258f 2024-09-22T01:01:07.254 INFO:teuthology.orchestra.run.smithi002.stdout:911 read 0x4600d42 thru 0x46079eb (0x6caa bytes) 2024-09-22T01:01:07.254 INFO:teuthology.orchestra.run.smithi002.stdout:913 read 0x6b0d05 thru 0x6b0dda (0xd6 bytes) 2024-09-22T01:01:07.255 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.255 INFO:teuthology.orchestra.run.smithi002.stdout:915 read 0x6329350 thru 0x6331e64 (0x8b15 bytes) 2024-09-22T01:01:07.255 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.255 INFO:teuthology.orchestra.run.smithi002.stdout:917 trunc from 0x7b3258f to 0x7a02f54 2024-09-22T01:01:07.261 INFO:teuthology.orchestra.run.smithi002.stdout:918 punch from 0x6cbe1e3 to 0x6cc7bb1, (0x99ce bytes) 2024-09-22T01:01:07.261 INFO:teuthology.orchestra.run.smithi002.stdout:919 read 0x487b8c8 thru 0x4889f3a (0xe673 bytes) 2024-09-22T01:01:07.261 INFO:teuthology.orchestra.run.smithi002.stdout:921 write 0x15b639d thru 0x15c213b (0xbd9f bytes) 2024-09-22T01:01:07.264 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.265 INFO:teuthology.orchestra.run.smithi002.stdout:927 write 0x450df2e thru 0x45156cb (0x779e bytes) 2024-09-22T01:01:07.275 INFO:teuthology.orchestra.run.smithi002.stdout:928 writesame 0x8a33bda thru 0x8a349e0 data_size 0x1b(0xe07 bytes) 2024-09-22T01:01:07.275 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.275 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.275 INFO:teuthology.orchestra.run.smithi002.stdout:932 punch from 0x18074ae to 0x180f2aa, (0x7dfc bytes) 2024-09-22T01:01:07.275 INFO:teuthology.orchestra.run.smithi002.stdout:933 read 0x802a25 thru 0x806ad8 (0x40b4 bytes) 2024-09-22T01:01:07.276 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.276 INFO:teuthology.orchestra.run.smithi002.stdout:935 punch from 0x738cc1e to 0x738cd1c, (0xfe bytes) 2024-09-22T01:01:07.282 INFO:teuthology.orchestra.run.smithi002.stdout:936 trunc from 0x8a349e1 to 0x44a1d63 2024-09-22T01:01:07.283 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:07.283 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:07.296 INFO:teuthology.orchestra.run.smithi002.stdout:938 write 0x26520dc thru 0x265dd4d (0xbc72 bytes) 2024-09-22T01:01:07.298 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:07.299 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.299 INFO:teuthology.orchestra.run.smithi002.stdout:940 read 0x366683a thru 0x367459d (0xdd64 bytes) 2024-09-22T01:01:07.306 INFO:teuthology.orchestra.run.smithi002.stdout:942 trunc from 0x44a1d63 to 0x879afd9 2024-09-22T01:01:07.312 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.312 INFO:teuthology.orchestra.run.smithi002.stdout:945 read 0x329384c thru 0x3299ae9 (0x629e bytes) 2024-09-22T01:01:07.315 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:07.319 INFO:teuthology.orchestra.run.smithi002.stdout:946 trunc from 0x879afd9 to 0x39f0020 2024-09-22T01:01:07.328 INFO:teuthology.orchestra.run.smithi002.stdout:947 read 0x2dfab21 thru 0x2e08567 (0xda47 bytes) 2024-09-22T01:01:07.328 INFO:teuthology.orchestra.run.smithi002.stdout:949 read 0xad5e58 thru 0xadec2a (0x8dd3 bytes) 2024-09-22T01:01:07.344 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:07.345+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:07.348 INFO:teuthology.orchestra.run.smithi002.stdout:952 write 0xcfdba08 thru 0xcfe4e99 (0x9492 bytes) 2024-09-22T01:01:07.352 INFO:teuthology.orchestra.run.smithi002.stdout:954 writesame 0x57ca55 thru 0x57ec71 data_size 0x3(0x221d bytes) 2024-09-22T01:01:07.355 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.355 INFO:teuthology.orchestra.run.smithi002.stdout:956 write 0x625efe4 thru 0x62638a6 (0x48c3 bytes) 2024-09-22T01:01:07.360 INFO:teuthology.orchestra.run.smithi002.stdout:957 punch from 0x762ceab to 0x763bff2, (0xf147 bytes) 2024-09-22T01:01:07.360 INFO:teuthology.orchestra.run.smithi002.stdout:958 write 0xb58408e thru 0xb585028 (0xf9b bytes) 2024-09-22T01:01:07.363 INFO:teuthology.orchestra.run.smithi002.stdout:959 read 0x24e965c thru 0x24ee40b (0x4db0 bytes) 2024-09-22T01:01:07.364 INFO:teuthology.orchestra.run.smithi002.stdout:960 read 0x5885629 thru 0x588b452 (0x5e2a bytes) 2024-09-22T01:01:07.364 INFO:teuthology.orchestra.run.smithi002.stdout:962 read 0x574e0c1 thru 0x574ee2e (0xd6e bytes) 2024-09-22T01:01:07.364 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:07.364 INFO:teuthology.orchestra.run.smithi002.stdout:964 clone 9 order 25 su 1048576 sc 14 2024-09-22T01:01:07.397 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-22T01:01:07.445 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:07.444+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:07.466 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:07.466 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:07.470 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:07.470 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:07.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:07.485 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:07.501 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:07.502 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:07.520 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:07.520 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:07.535 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:07.533+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:07.540 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:07.540 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:07.541 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:07.559 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:07.580 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:07.580 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:07.583 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:07.583 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:07.607 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-22T01:01:07.609 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:07.609 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:07.609 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:07.609 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:07.609 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:07.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:07.649 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:07.648+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:07.729 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:07.728+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:07.790 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:07.790 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:07.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2955 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.411061966, 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044281583999999999, 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2955 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378128+0000", 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378131+0000", 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.864e-06 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378137+0000", 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8020000000000003e-06 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378141+0000", 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422313+0000", 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044172553000000003 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422359+0000", 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6146000000000002e-05 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422409+0000", 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9576000000000001e-05 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2956 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3284992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:07.795 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.411032278, 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044364010000000002, 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2956 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378159+0000", 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8679999999999999e-06 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378164+0000", 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2499999999999997e-06 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378169+0000", 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3900000000000003e-06 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422438+0000", 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044268764000000002 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422478+0000", 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.066e-05 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422521+0000", 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2741999999999998e-05 2024-09-22T01:01:07.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2957 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.411015557000001, 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044474276, 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2957 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1299999999999999e-07 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378175+0000", 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3039999999999999e-06 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378179+0000", 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.388e-06 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378188+0000", 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0569999999999999e-06 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422550+0000", 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044362523000000001 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422597+0000", 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6493999999999999e-05 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422648+0000", 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1297e-05 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2958 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410995510999999, 2024-09-22T01:01:07.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044568617999999997, 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2958 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.75e-07 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378196+0000", 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4959999999999999e-06 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378199+0000", 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7969999999999998e-06 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378202+0000", 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.65e-06 2024-09-22T01:01:07.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422678+0000", 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044475564000000002 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422718+0000", 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0825000000000003e-05 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422762+0000", 2024-09-22T01:01:07.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3810999999999998e-05 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2959 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410983254, 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044681193000000001, 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2959 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378207+0000", 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.316e-06 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378211+0000", 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378213+0000", 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.588e-06 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422794+0000", 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044580928999999998 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422840+0000", 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5231000000000001e-05 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422887+0000", 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7543999999999997e-05 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2960 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410972209000001, 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044780915999999997, 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.813 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2960 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378218+0000", 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.032e-06 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378222+0000", 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4690000000000002e-06 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378224+0000", 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5040000000000001e-06 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422915+0000", 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044691036000000003 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422956+0000", 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0881999999999999e-05 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422998+0000", 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1782000000000001e-05 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2961 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:07.817 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410961587999999, 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891911, 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2961 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:07.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378229+0000", 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.071e-06 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378232+0000", 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378235+0000", 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4609999999999998e-06 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423029+0000", 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044794498000000002 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423072+0000", 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2796000000000002e-05 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423120+0000", 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7574999999999998e-05 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2962 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410951057, 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044991058, 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2962 2024-09-22T01:01:07.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:07.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378239+0000", 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378243+0000", 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3459999999999998e-06 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378245+0000", 2024-09-22T01:01:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6340000000000001e-06 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423149+0000", 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044903169999999999 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423188+0000", 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8974999999999999e-05 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423229+0000", 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1655e-05 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2963 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410940445, 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104404000000001, 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:07.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378250+0000", 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.001e-06 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378255+0000", 2024-09-22T01:01:07.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7949999999999998e-06 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378264+0000", 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6129999999999993e-06 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423259+0000", 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044994348000000003 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423302+0000", 2024-09-22T01:01:07.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3733000000000003e-05 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423353+0000", 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0689000000000002e-05 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2964 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410920157, 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045200033000000001, 2024-09-22T01:01:07.830 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2964 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378275+0000", 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3850000000000001e-06 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378279+0000", 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378281+0000", 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5129999999999999e-06 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423382+0000", 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045100385 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423422+0000", 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0265999999999997e-05 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423469+0000", 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7110999999999997e-05 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.834 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2965 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410897649000001, 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045300746000000003, 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2965 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6899999999999999e-07 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378294+0000", 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.951e-06 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378298+0000", 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1239999999999998e-06 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378301+0000", 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.779e-06 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423497+0000", 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045196473000000001 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423544+0000", 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6622000000000002e-05 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423592+0000", 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8528e-05 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.838 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2966 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410884251000001, 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045402558000000003, 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2966 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378306+0000", 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2869999999999999e-06 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378310+0000", 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.258e-06 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378312+0000", 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7810000000000001e-06 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423622+0000", 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045309464000000001 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423663+0000", 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1310999999999998e-05 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423708+0000", 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4248000000000001e-05 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3013 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410873154000001, 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045530159000000001, 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.843 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3013 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378317+0000", 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378321+0000", 2024-09-22T01:01:07.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3110000000000001e-06 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378323+0000", 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5600000000000001e-06 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423737+0000", 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045414035999999998 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423781+0000", 2024-09-22T01:01:07.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3279999999999999e-05 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423846+0000", 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5634000000000002e-05 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3014 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410862496, 2024-09-22T01:01:07.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045653157, 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3014 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378715+0000", 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038801599999999998 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378719+0000", 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4039999999999997e-06 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378731+0000", 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1755e-05 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378739+0000", 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.712e-06 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423876+0000", 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045137505000000001 2024-09-22T01:01:07.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423918+0000", 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2339999999999998e-05 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423980+0000", 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1425000000000002e-05 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.851 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3015 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410435121999999, 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045369596999999998, 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3015 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378755+0000", 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0100000000000003e-07 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378759+0000", 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378766+0000", 2024-09-22T01:01:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.029e-06 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378769+0000", 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7189999999999999e-06 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424013+0000", 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045243798000000002 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424059+0000", 2024-09-22T01:01:07.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5725e-05 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424124+0000", 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4851999999999995e-05 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3016 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:07.856 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.410413723, 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045478877000000001, 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3016 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379160+0000", 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038423800000000002 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379164+0000", 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379173+0000", 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.439e-06 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379178+0000", 2024-09-22T01:01:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3909999999999997e-06 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424155+0000", 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044976781 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424198+0000", 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769000000000003e-05 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424254+0000", 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6641000000000002e-05 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3017 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.409996683999999, 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045202985000000001, 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3017 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:07.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9400000000000005e-07 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379198+0000", 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379215+0000", 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6863000000000001e-05 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379219+0000", 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0749999999999999e-06 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424286+0000", 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045067013000000003 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424332+0000", 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5531000000000002e-05 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424395+0000", 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3740999999999998e-05 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3018 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:07.865 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.409960931000001, 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045293472000000001, 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3018 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379430+0000", 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020146699999999999 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379435+0000", 2024-09-22T01:01:07.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379443+0000", 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4819999999999994e-06 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379461+0000", 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.753e-05 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424424+0000", 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044962716999999999 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424464+0000", 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0824000000000001e-05 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424522+0000", 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7383999999999998e-05 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.869 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3019 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.409718675000001, 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045194026999999998, 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3019 2024-09-22T01:01:07.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381690+0000", 2024-09-22T01:01:07.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022191849999999998 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381696+0000", 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3860000000000004e-06 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381705+0000", 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2480000000000002e-06 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381712+0000", 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.579e-06 2024-09-22T01:01:07.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424554+0000", 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042842327999999999 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424601+0000", 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6697999999999999e-05 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424665+0000", 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3602999999999994e-05 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.873 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:8990 3.a 3:51e071dd:::rbd_data.117cc87baa96.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.608172776, 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048973669999999997, 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:07.874 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 8990 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:07.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181019+0000", 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4889999999999998e-06 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181038+0000", 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9097e-05 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181066+0000", 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7514000000000001e-05 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:07.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181078+0000", 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2626000000000001e-05 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181196+0000", 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118208 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181388+0000", 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019203100000000001 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:07.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.188674+0000", 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072856120000000003 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229668+0000", 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040994282 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229764+0000", 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6036e-05 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229859+0000", 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5088999999999997e-05 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229990+0000", 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130686 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:07.879 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:07.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-22T01:01:07.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:07.882 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:01:07.883 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:07.876+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:07.981 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone8 intact 2024-09-22T01:01:07.981 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:07.980+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:08.016 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:08.016 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:08.016 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:08.017 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:08.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:08.082 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:08.080+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:08.124 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-22T01:01:08.153 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:08.153 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:08.153 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:08.153 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:08.153 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:08.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:08.209 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:08.208+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:08.219 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-22T01:01:08.221 INFO:teuthology.orchestra.run.smithi002.stdout:965 punch from 0xccbb8ef to 0xccc7b12, (0xc223 bytes) 2024-09-22T01:01:08.227 INFO:teuthology.orchestra.run.smithi002.stdout:968 punch from 0x906c073 to 0x9076efe, (0xae8b bytes) 2024-09-22T01:01:08.227 INFO:teuthology.orchestra.run.smithi002.stdout:969 read 0x77af2ba thru 0x77b1321 (0x2068 bytes) 2024-09-22T01:01:08.227 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.227 INFO:teuthology.orchestra.run.smithi002.stdout:973 punch from 0x9b5584c to 0x9b6083c, (0xaff0 bytes) 2024-09-22T01:01:08.227 INFO:teuthology.orchestra.run.smithi002.stdout:974 write 0x5e76e3d thru 0x5e7ee7b (0x803f bytes) 2024-09-22T01:01:08.234 INFO:teuthology.orchestra.run.smithi002.stdout:975 writesame 0x2d25bfb thru 0x2d2dcf2 data_size 0x4(0x80f8 bytes) 2024-09-22T01:01:08.244 INFO:teuthology.orchestra.run.smithi002.stdout:976 write 0x9d0afee thru 0x9d0c5e2 (0x15f5 bytes) 2024-09-22T01:01:08.246 INFO:teuthology.orchestra.run.smithi002.stdout:977 read 0xa3fd0bc thru 0xa4012f1 (0x4236 bytes) 2024-09-22T01:01:08.246 INFO:teuthology.orchestra.run.smithi002.stdout:978 write 0x4344fca thru 0x434bdb1 (0x6de8 bytes) 2024-09-22T01:01:08.253 INFO:teuthology.orchestra.run.smithi002.stdout:979 read 0x1b49bdc thru 0x1b522cc (0x86f1 bytes) 2024-09-22T01:01:08.253 INFO:teuthology.orchestra.run.smithi002.stdout:981 write 0x1cde8b6 thru 0x1ceb8c6 (0xd011 bytes) 2024-09-22T01:01:08.262 INFO:teuthology.orchestra.run.smithi002.stdout:982 punch from 0x3b580d1 to 0x3b5c8f2, (0x4821 bytes) 2024-09-22T01:01:08.262 INFO:teuthology.orchestra.run.smithi002.stdout:983 punch from 0x9efe6f4 to 0x9f0899c, (0xa2a8 bytes) 2024-09-22T01:01:08.262 INFO:teuthology.orchestra.run.smithi002.stdout:984 punch from 0xb62a012 to 0xb62b7b3, (0x17a1 bytes) 2024-09-22T01:01:08.262 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.262 INFO:teuthology.orchestra.run.smithi002.stdout:986 read 0x114ba93 thru 0x115073a (0x4ca8 bytes) 2024-09-22T01:01:08.263 INFO:teuthology.orchestra.run.smithi002.stdout:987 write 0x3ee5ba7 thru 0x3ef52b3 (0xf70d bytes) 2024-09-22T01:01:08.270 INFO:teuthology.orchestra.run.smithi002.stdout:988 writesame 0xb79ea35 thru 0xb7a9276 data_size 0x9(0xa842 bytes) 2024-09-22T01:01:08.279 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.279 INFO:teuthology.orchestra.run.smithi002.stdout:992 read 0xc10135e thru 0xc10c29b (0xaf3e bytes) 2024-09-22T01:01:08.279 INFO:teuthology.orchestra.run.smithi002.stdout:993 writesame 0xc8912d6 thru 0xc893cea data_size 0x39(0x2a15 bytes) 2024-09-22T01:01:08.283 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:08.280+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:08.285 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.285 INFO:teuthology.orchestra.run.smithi002.stdout:996 write 0x115cf2d thru 0x11695cd (0xc6a1 bytes) 2024-09-22T01:01:08.287 INFO:teuthology.orchestra.run.smithi002.stdout:997 read 0x576e63a thru 0x577be57 (0xd81e bytes) 2024-09-22T01:01:08.288 INFO:teuthology.orchestra.run.smithi002.stdout:998 write 0x40f9cc0 thru 0x41092c6 (0xf607 bytes) 2024-09-22T01:01:08.296 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:08.302 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:08.302 INFO:tasks.thrashosds.thrasher:Failed to reset_purged_snaps_last, ignoring 2024-09-22T01:01:08.303 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-22T01:01:08.306 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.414650109, 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:01:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:01:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1907 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094113+0000", 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.211751691, 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056039765999999998, 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1907 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094113+0000", 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094113+0000", 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094115+0000", 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.457e-06 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094122+0000", 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094141+0000", 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.908e-05 2024-09-22T01:01:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094155+0000", 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4276e-05 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.149936+0000", 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055781612000000001 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150045+0000", 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108498 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150153+0000", 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108087 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1908 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1671168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094173+0000", 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.211691649, 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056552275999999999, 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1908 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094173+0000", 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094173+0000", 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094175+0000", 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.24e-06 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094180+0000", 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1370000000000001e-06 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094197+0000", 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.717e-05 2024-09-22T01:01:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094209+0000", 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2106999999999999e-05 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150243+0000", 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056034349999999997 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150464+0000", 2024-09-22T01:01:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022080400000000001 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150726+0000", 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000261468 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1909 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094226+0000", 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.211638609, 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056756131000000001, 2024-09-22T01:01:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1909 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094226+0000", 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094226+0000", 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094227+0000", 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.037e-06 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094233+0000", 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9830000000000001e-06 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094251+0000", 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7367000000000002e-05 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094266+0000", 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4763000000000001e-05 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150808+0000", 2024-09-22T01:01:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056542800999999997 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150891+0000", 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2942999999999995e-05 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.150983+0000", 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1236999999999994e-05 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1910 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094284+0000", 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.211580996, 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056887912999999998, 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1910 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094284+0000", 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094284+0000", 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094285+0000", 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:01:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094291+0000", 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.643e-06 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094308+0000", 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.766e-05 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094341+0000", 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2193000000000002e-05 2024-09-22T01:01:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151027+0000", 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056685985000000001 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151097+0000", 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0366999999999999e-05 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151172+0000", 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4998000000000003e-05 2024-09-22T01:01:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1911 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 1941504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094388+0000", 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.211476510000001, 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056956143000000001, 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1911 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094388+0000", 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094388+0000", 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094390+0000", 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1769999999999999e-06 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094397+0000", 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094418+0000", 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0659999999999999e-05 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094433+0000", 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4946000000000001e-05 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151212+0000", 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056779863999999999 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151266+0000", 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3118000000000002e-05 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151345+0000", 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9005000000000007e-05 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1912 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094453+0000", 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.211412357, 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.057051945, 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1912 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094453+0000", 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094453+0000", 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094701+0000", 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000248454 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094710+0000", 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5099999999999998e-06 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094732+0000", 2024-09-22T01:01:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2557000000000001e-05 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094749+0000", 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6739999999999999e-05 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151381+0000", 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056631853000000003 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151434+0000", 2024-09-22T01:01:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2967000000000003e-05 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151505+0000", 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0864000000000005e-05 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1913 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2121728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094773+0000", 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.211092366000001, 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056874896000000001, 2024-09-22T01:01:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1913 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094773+0000", 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094773+0000", 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094774+0000", 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3859999999999999e-06 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094798+0000", 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4326999999999999e-05 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094819+0000", 2024-09-22T01:01:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0573000000000001e-05 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094834+0000", 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4966e-05 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151537+0000", 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056702874 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151582+0000", 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4827000000000003e-05 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151648+0000", 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5943000000000002e-05 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1914 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.094853+0000", 2024-09-22T01:01:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.211011724, 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056940442000000001, 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1914 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094853+0000", 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.094853+0000", 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096093+0000", 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012396900000000001 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096102+0000", 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1570000000000006e-06 2024-09-22T01:01:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096129+0000", 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6622999999999999e-05 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096147+0000", 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8114e-05 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151680+0000", 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055532893 2024-09-22T01:01:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151726+0000", 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6233000000000003e-05 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.151794+0000", 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7731999999999995e-05 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1918 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2572288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096474+0000", 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.209391476, 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056054800000000002, 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1918 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096474+0000", 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096474+0000", 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096475+0000", 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096481+0000", 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096499+0000", 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8332999999999999e-05 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096512+0000", 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.346e-05 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152430+0000", 2024-09-22T01:01:08.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055917426999999999 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152478+0000", 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7988999999999997e-05 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152528+0000", 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0504999999999998e-05 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1919 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2662400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096530+0000", 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.209335477, 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056125652999999998, 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1919 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096530+0000", 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096530+0000", 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096531+0000", 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096536+0000", 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096554+0000", 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.732e-05 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096584+0000", 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0762999999999997e-05 2024-09-22T01:01:08.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152561+0000", 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055976763999999998 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152607+0000", 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5405999999999999e-05 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152655+0000", 2024-09-22T01:01:08.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8622999999999998e-05 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1920 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096603+0000", 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.209261964, 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056177865, 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1920 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096603+0000", 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096603+0000", 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096604+0000", 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096610+0000", 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096628+0000", 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8369999999999999e-05 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096642+0000", 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.415e-05 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152687+0000", 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056044683999999997 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152732+0000", 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4613999999999997e-05 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152781+0000", 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9296999999999999e-05 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1921 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2842624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096659+0000", 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.209205733999999, 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056245989000000003, 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1921 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096659+0000", 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096659+0000", 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096661+0000", 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.246e-06 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096666+0000", 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2340000000000002e-06 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096683+0000", 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7493e-05 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096697+0000", 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3859999999999999e-05 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152813+0000", 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056116054999999998 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152858+0000", 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4690000000000001e-05 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152905+0000", 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7410999999999997e-05 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.363 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1922 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 2932736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096714+0000", 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.209150762, 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056318841000000001, 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1922 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096714+0000", 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096714+0000", 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096715+0000", 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0270000000000001e-06 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096720+0000", 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096738+0000", 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7105999999999999e-05 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096751+0000", 2024-09-22T01:01:08.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3617e-05 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152936+0000", 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056185156999999999 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.152984+0000", 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.774e-05 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153033+0000", 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9044e-05 2024-09-22T01:01:08.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1923 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3022848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096768+0000", 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.209096539000001, 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056392314999999998, 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1923 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096768+0000", 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096768+0000", 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096770+0000", 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.097e-06 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096775+0000", 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1070000000000004e-06 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096792+0000", 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7706e-05 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096806+0000", 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153066+0000", 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056260403 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153113+0000", 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6464000000000001e-05 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153161+0000", 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7994000000000002e-05 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1924 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096823+0000", 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.209042135000001, 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056462601000000001, 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.372 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1924 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096823+0000", 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096823+0000", 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096824+0000", 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.003e-06 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096845+0000", 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1376e-05 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096864+0000", 2024-09-22T01:01:08.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8810000000000001e-05 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096878+0000", 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4136e-05 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153192+0000", 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056314125 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153237+0000", 2024-09-22T01:01:08.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4721000000000001e-05 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153285+0000", 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8430000000000002e-05 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1925 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3203072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096897+0000", 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.208968483, 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056516714000000003, 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.376 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1925 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096897+0000", 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096897+0000", 2024-09-22T01:01:08.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096898+0000", 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096904+0000", 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6069999999999998e-06 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096921+0000", 2024-09-22T01:01:08.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7799999999999999e-05 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096935+0000", 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3983e-05 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153317+0000", 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056381477999999999 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153365+0000", 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153413+0000", 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7839000000000001e-05 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1926 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.096953+0000", 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.208912117000001, 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056584061999999997, 2024-09-22T01:01:08.380 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1926 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096953+0000", 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.096953+0000", 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097198+0000", 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024509500000000002 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097207+0000", 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0939999999999997e-06 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097229+0000", 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2436e-05 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097246+0000", 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6603e-05 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153444+0000", 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056197902000000001 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153489+0000", 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4706000000000002e-05 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153537+0000", 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8226000000000002e-05 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1927 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3383296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.097275+0000", 2024-09-22T01:01:08.384 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.208589803000001, 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056399983000000001, 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1927 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097275+0000", 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097275+0000", 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097875+0000", 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059935100000000003 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097893+0000", 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8386e-05 2024-09-22T01:01:08.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097918+0000", 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4674000000000001e-05 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.097953+0000", 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5311000000000001e-05 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153569+0000", 2024-09-22T01:01:08.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.055615708 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153614+0000", 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4966999999999998e-05 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.153675+0000", 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1586000000000003e-05 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:1931 3.e 3:744ea144:::rbd_data.117cf538ee01.000000000000001e:3 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.098137+0000", 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.207728137, 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.056042361999999998, 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1931 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.098137+0000", 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.098137+0000", 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.101915+0000", 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003778563 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.101927+0000", 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1576999999999999e-05 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.101954+0000", 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7073999999999999e-05 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.101973+0000", 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8893999999999999e-05 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.154087+0000", 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.052113605 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.154132+0000", 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5061000000000001e-05 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.154179+0000", 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7587999999999999e-05 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.393 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:08.393 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:08.393 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:08.395 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:08.393+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:08.395 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:999 trunc from 0xcfe4e9a to 0x1e33b50 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1000 punch from 0xbbf4b9 to 0xbc4d26, (0x586d bytes) 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1001 punch from 0x52f59f to 0x53dfea, (0xea4b bytes) 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1003 read 0xd12543 thru 0xd207f2 (0xe2b0 bytes) 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1004 read 0xc4fc78 thru 0xc5e83d (0xebc6 bytes) 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1005 trunc from 0x1e33b50 to 0x4e1850e 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1007 read 0x257a441 thru 0x2584ffc (0xabbc bytes) 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1009 read 0x1c509e thru 0x1c5607 (0x56a bytes) 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1010 punch from 0x4128783 to 0x412f027, (0x68a4 bytes) 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1011 writesame 0xcbfdb6d thru 0xcc0b930 data_size 0x72(0xddc4 bytes) 2024-09-22T01:01:08.397 INFO:teuthology.orchestra.run.smithi002.stdout:1012 writesame 0x84fd502 thru 0x84ffce8 data_size 0xf(0x27e7 bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1015 write 0xccfa267 thru 0xcd07eb7 (0xdc51 bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1016 read 0xad4335d thru 0xad43398 (0x3c bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1018 punch from 0x978976a to 0x978f218, (0x5aae bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1019 punch from 0xb5f7af8 to 0xb600497, (0x899f bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1020 write 0x316a05d thru 0x31709ec (0x6990 bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1021 write 0xd20db4b thru 0xd210306 (0x27bc bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1022 read 0x29110e6 thru 0x291d45a (0xc375 bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1025 write 0x18cbd12 thru 0x18d5135 (0x9424 bytes) 2024-09-22T01:01:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:1026 write 0x3ccad75 thru 0x3cd1f3e (0x71ca bytes) 2024-09-22T01:01:08.399 INFO:teuthology.orchestra.run.smithi002.stdout:1027 trunc from 0xd210307 to 0xa31c029 2024-09-22T01:01:08.419 INFO:teuthology.orchestra.run.smithi002.stdout:1028 trunc from 0xa31c029 to 0x31f407 2024-09-22T01:01:08.495 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:08.492+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:08.497 INFO:teuthology.orchestra.run.smithi002.stdout:1029 write 0x820a159 thru 0x8214468 (0xa310 bytes) 2024-09-22T01:01:08.505 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.505 INFO:teuthology.orchestra.run.smithi002.stdout:1032 write 0x38bb3a4 thru 0x38c19d0 (0x662d bytes) 2024-09-22T01:01:08.508 INFO:teuthology.orchestra.run.smithi002.stdout:1033 write 0x19b18e4 thru 0x19b3588 (0x1ca5 bytes) 2024-09-22T01:01:08.527 INFO:teuthology.orchestra.run.smithi002.stdout:1034 write 0xaad1f7e thru 0xaad52c3 (0x3346 bytes) 2024-09-22T01:01:08.530 INFO:teuthology.orchestra.run.smithi002.stdout:1035 punch from 0x1106583 to 0x110a373, (0x3df0 bytes) 2024-09-22T01:01:08.530 INFO:teuthology.orchestra.run.smithi002.stdout:1037 writesame 0xc8cd8f thru 0xc8ef98 data_size 0x1(0x220a bytes) 2024-09-22T01:01:08.545 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.545 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.545 INFO:teuthology.orchestra.run.smithi002.stdout:1040 read 0x2602d2 thru 0x261375 (0x10a4 bytes) 2024-09-22T01:01:08.548 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:08.548 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:08.560 INFO:teuthology.orchestra.run.smithi002.stdout:1041 writesame 0xe1b472b thru 0xe1c35f6 data_size 0x74(0xeecc bytes) 2024-09-22T01:01:08.562 INFO:teuthology.orchestra.run.smithi002.stdout:1042 write 0xe147cf3 thru 0xe1513a3 (0x96b1 bytes) 2024-09-22T01:01:08.564 INFO:teuthology.orchestra.run.smithi002.stdout:1043 punch from 0x3464649 to 0x346cad7, (0x848e bytes) 2024-09-22T01:01:08.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:08.567 INFO:teuthology.orchestra.run.smithi002.stdout:1044 trunc from 0xe1c35f7 to 0xca05911 2024-09-22T01:01:08.583 INFO:teuthology.orchestra.run.smithi002.stdout:1047 trunc from 0xca05911 to 0xaa54118 2024-09-22T01:01:08.587 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-22T01:01:08.600 INFO:teuthology.orchestra.run.smithi002.stdout:1049 trunc from 0xaa54118 to 0xdf88d2b 2024-09-22T01:01:08.600 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:08.596+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:08.609 INFO:teuthology.orchestra.run.smithi002.stdout:1050 write 0x7652a2e thru 0x7654fda (0x25ad bytes) 2024-09-22T01:01:08.614 INFO:teuthology.orchestra.run.smithi002.stdout:1052 punch from 0x27aa56d to 0x27b2c1b, (0x86ae bytes) 2024-09-22T01:01:08.634 INFO:teuthology.orchestra.run.smithi002.stdout:1054 trunc from 0xdf88d2b to 0x7a090d 2024-09-22T01:01:08.683 INFO:teuthology.orchestra.run.smithi002.stdout:1055 writesame 0xd245a09 thru 0xd2468be data_size 0xe(0xeb6 bytes) 2024-09-22T01:01:08.686 INFO:teuthology.orchestra.run.smithi002.stdout:1056 write 0x7ba9f7e thru 0x7bb65e8 (0xc66b bytes) 2024-09-22T01:01:08.692 INFO:teuthology.orchestra.run.smithi002.stdout:1058 punch from 0x8a3098 to 0x8b2a79, (0xf9e1 bytes) 2024-09-22T01:01:08.695 INFO:teuthology.orchestra.run.smithi002.stdout:1059 trunc from 0xd2468bf to 0xaea78d0 2024-09-22T01:01:08.697 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:08.696+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:08.700 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:08.700 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:08.708 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.710 INFO:teuthology.orchestra.run.smithi002.stdout:1064 trunc from 0xaea78d0 to 0xc327c03 2024-09-22T01:01:08.719 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:08.719 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.719 INFO:teuthology.orchestra.run.smithi002.stdout:1066 write 0x73205d1 thru 0x7321748 (0x1178 bytes) 2024-09-22T01:01:08.721 INFO:teuthology.orchestra.run.smithi002.stdout:1067 writesame 0xb05b93c thru 0xb0676e9 data_size 0x3(0xbdae bytes) 2024-09-22T01:01:08.743 INFO:teuthology.orchestra.run.smithi002.stdout:1068 write 0x8114f70 thru 0x811d910 (0x89a1 bytes) 2024-09-22T01:01:08.745 INFO:teuthology.orchestra.run.smithi002.stdout:1069 read 0x428a123 thru 0x4295f1a (0xbdf8 bytes) 2024-09-22T01:01:08.755 INFO:teuthology.orchestra.run.smithi002.stdout:1070 write 0xcec723f thru 0xced3d85 (0xcb47 bytes) 2024-09-22T01:01:08.763 INFO:teuthology.orchestra.run.smithi002.stdout:1071 writesame 0x3b350e2 thru 0x3b37e3d data_size 0x2(0x2d5c bytes) 2024-09-22T01:01:08.770 INFO:teuthology.orchestra.run.smithi002.stdout:1072 read 0x96a1ada thru 0x96a630d (0x4834 bytes) 2024-09-22T01:01:08.771 INFO:teuthology.orchestra.run.smithi002.stdout:1073 punch from 0xbfe9daa to 0xbff0711, (0x6967 bytes) 2024-09-22T01:01:08.771 INFO:teuthology.orchestra.run.smithi002.stdout:1075 read 0x2faf64 thru 0x304bbb (0x9c58 bytes) 2024-09-22T01:01:08.772 INFO:teuthology.orchestra.run.smithi002.stdout:1077 read 0x2ec4175 thru 0x2ed3505 (0xf391 bytes) 2024-09-22T01:01:08.773 INFO:teuthology.orchestra.run.smithi002.stdout:1078 writesame 0x8de1c80 thru 0x8de5744 data_size 0x33(0x3ac5 bytes) 2024-09-22T01:01:08.776 INFO:teuthology.orchestra.run.smithi002.stdout:1081 writesame 0x7bd91b4 thru 0x7bde604 data_size 0x5(0x5451 bytes) 2024-09-22T01:01:08.779 INFO:teuthology.orchestra.run.smithi002.stdout:1082 write 0x8ab039 thru 0x8b2efd (0x7ec5 bytes) 2024-09-22T01:01:08.782 INFO:teuthology.orchestra.run.smithi002.stdout:1083 read 0x790c1c thru 0x79d2c1 (0xc6a6 bytes) 2024-09-22T01:01:08.782 INFO:teuthology.orchestra.run.smithi002.stdout:1084 write 0x37c4167 thru 0x37cb4bc (0x7356 bytes) 2024-09-22T01:01:08.798 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:08.797+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:08.801 INFO:teuthology.orchestra.run.smithi002.stdout:1085 trunc from 0xced3d86 to 0x37a83c9 2024-09-22T01:01:08.813 INFO:teuthology.orchestra.run.smithi002.stdout:1086 read 0x334a2e thru 0x339985 (0x4f58 bytes) 2024-09-22T01:01:08.822 INFO:teuthology.orchestra.run.smithi002.stdout:1088 trunc from 0x37a83c9 to 0x94c7874 2024-09-22T01:01:08.843 INFO:teuthology.orchestra.run.smithi002.stdout:1090 writesame 0xae470aa thru 0xae50fce data_size 0x1(0x9f25 bytes) 2024-09-22T01:01:08.852 INFO:teuthology.orchestra.run.smithi002.stdout:1091 writesame 0xaf6fdca thru 0xaf7e129 data_size 0x44(0xe360 bytes) 2024-09-22T01:01:08.852 INFO:teuthology.orchestra.run.smithi002.stdout:1092 writesame 0x6fe66f9 thru 0x6fe81d0 data_size 0x4(0x1ad8 bytes) 2024-09-22T01:01:08.854 INFO:teuthology.orchestra.run.smithi002.stdout:1093 write 0x28e0cc5 thru 0x28ec3dd (0xb719 bytes) 2024-09-22T01:01:08.858 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:08.858 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:08.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:08.858 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:08.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.858 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8692 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.858 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7679593219999994, 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01097323, 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8692 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:08.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089896+0000", 2024-09-22T01:01:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089904+0000", 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3440000000000001e-06 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089911+0000", 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5799999999999997e-06 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100463+0000", 2024-09-22T01:01:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010552713 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100732+0000", 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026876599999999998 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100866+0000", 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133623 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8693 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7679322820000003, 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011283726000000001, 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8693 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:01:08.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089924+0000", 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089932+0000", 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9929999999999995e-06 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089941+0000", 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9719999999999998e-06 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100989+0000", 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011048231 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101092+0000", 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102975 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101203+0000", 2024-09-22T01:01:08.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111648 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8712 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 630784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7678313190000008, 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011403979999999999, 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8712 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090022+0000", 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.19e-06 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090027+0000", 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.604e-06 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090041+0000", 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3114e-05 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090049+0000", 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6479999999999992e-06 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101235+0000", 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011185428000000001 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101309+0000", 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4376999999999996e-05 2024-09-22T01:01:08.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101425+0000", 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011561900000000001 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8713 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7677870630000001, 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011573441, 2024-09-22T01:01:08.871 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8713 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090069+0000", 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9840000000000003e-06 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090077+0000", 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6639999999999998e-06 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090083+0000", 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101474+0000", 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011391007 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101548+0000", 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4555999999999996e-05 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101638+0000", 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0223000000000007e-05 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8737 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.875 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7677614340000005, 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011746902, 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8737 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090093+0000", 2024-09-22T01:01:08.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1619999999999998e-06 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090099+0000", 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2249999999999997e-06 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090104+0000", 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9520000000000002e-06 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101668+0000", 2024-09-22T01:01:08.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011563339000000001 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101743+0000", 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5796000000000004e-05 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101837+0000", 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4055999999999995e-05 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.879 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8747 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 729088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7677389120000004, 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011923398, 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8747 2024-09-22T01:01:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9900000000000001e-07 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090127+0000", 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3447e-05 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090134+0000", 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3799999999999996e-06 2024-09-22T01:01:08.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090139+0000", 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8119999999999998e-06 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101866+0000", 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011726752 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101938+0000", 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102036+0000", 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8041000000000005e-05 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8757 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7677039440000009, 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012330969000000001, 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8757 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090151+0000", 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090159+0000", 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4869999999999996e-06 2024-09-22T01:01:08.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090164+0000", 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3210000000000004e-06 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102065+0000", 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011901022000000001 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102381+0000", 2024-09-22T01:01:08.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031597199999999998 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102479+0000", 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8101000000000001e-05 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8767 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:08.888 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.767680296, 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012517782, 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8767 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090174+0000", 2024-09-22T01:01:08.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090181+0000", 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6320000000000002e-06 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090186+0000", 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6949999999999999e-06 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102511+0000", 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012325469 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102601+0000", 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9802000000000005e-05 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102689+0000", 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8571000000000003e-05 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8777 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7676556679999997, 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012693987, 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8777 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090197+0000", 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090199+0000", 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090206+0000", 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4089999999999997e-06 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090212+0000", 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4380000000000003e-06 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102718+0000", 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012506735999999999 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102793+0000", 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5054000000000003e-05 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102890+0000", 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6825999999999998e-05 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8787 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 786432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7663768270000002, 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011612116, 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8787 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.897 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091476+0000", 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.11e-06 2024-09-22T01:01:08.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091482+0000", 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091494+0000", 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2047e-05 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091503+0000", 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:01:08.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102918+0000", 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011414951 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102992+0000", 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3279000000000001e-05 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103087+0000", 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5662000000000007e-05 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.900 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8797 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 802816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7663326700000006, 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011753412, 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8797 2024-09-22T01:01:08.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091520+0000", 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5000000000000003e-07 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091536+0000", 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6214000000000001e-05 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091546+0000", 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7799999999999995e-06 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091559+0000", 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2911000000000001e-05 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103116+0000", 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011557207 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103188+0000", 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2418999999999999e-05 2024-09-22T01:01:08.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103273+0000", 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4331000000000003e-05 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8807 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 811008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7662803060000005, 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011888717, 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8807 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091576+0000", 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5540000000000002e-06 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091586+0000", 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0502999999999999e-05 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091593+0000", 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.861e-06 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103300+0000", 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011706467999999999 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103371+0000", 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1308000000000002e-05 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103460+0000", 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9395999999999996e-05 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8818 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 761856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7662483939999998, 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012045223000000001, 2024-09-22T01:01:08.909 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8818 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2200000000000004e-07 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091607+0000", 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5440000000000001e-06 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091615+0000", 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0169999999999996e-06 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091620+0000", 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-06 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103488+0000", 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011867353000000001 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103561+0000", 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3481999999999999e-05 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103649+0000", 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7495000000000002e-05 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8821 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7662223210000008, 2024-09-22T01:01:08.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012214868, 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8821 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3099999999999998e-07 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091633+0000", 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5770000000000001e-06 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091641+0000", 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091647+0000", 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1079999999999998e-06 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103676+0000", 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012029361000000001 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103747+0000", 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0587999999999996e-05 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103844+0000", 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7606e-05 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8822 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:08.917 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7661960190000006, 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01239211, 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8822 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091659+0000", 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5770000000000001e-06 2024-09-22T01:01:08.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091667+0000", 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3410000000000003e-06 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091673+0000", 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8270000000000003e-06 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103872+0000", 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012199162 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103951+0000", 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8512999999999999e-05 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104048+0000", 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7212000000000004e-05 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.921 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8824 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7661702619999993, 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012562043, 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8824 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091685+0000", 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5349999999999999e-06 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091693+0000", 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3019999999999995e-06 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091707+0000", 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4345000000000001e-05 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104080+0000", 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012372321 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104156+0000", 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6734000000000001e-05 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104244+0000", 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7363999999999999e-05 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.925 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8848 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7648046280000003, 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011398933, 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8848 2024-09-22T01:01:08.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093048+0000", 2024-09-22T01:01:08.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.136e-06 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093054+0000", 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6969999999999998e-06 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093074+0000", 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.95e-05 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093083+0000", 2024-09-22T01:01:08.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3479999999999993e-06 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104273+0000", 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011189747999999999 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104352+0000", 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9560000000000004e-05 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104446+0000", 2024-09-22T01:01:08.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3943999999999994e-05 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8849 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 892928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7647522739999992, 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011546511000000001, 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8849 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093104+0000", 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4010000000000001e-06 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093113+0000", 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1349999999999998e-06 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093119+0000", 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3720000000000001e-06 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104477+0000", 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011358157000000001 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104554+0000", 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6390000000000006e-05 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104646+0000", 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2379000000000004e-05 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8868 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 860160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7647221270000006, 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011726966, 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8868 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5999999999999999e-07 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093133+0000", 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6900000000000001e-06 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093142+0000", 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6780000000000006e-06 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093148+0000", 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1160000000000004e-06 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104677+0000", 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011529285 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104756+0000", 2024-09-22T01:01:08.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104857+0000", 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010123099999999999 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8869 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:08.938 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7646888250000003, 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011914377, 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8869 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093164+0000", 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093167+0000", 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4359999999999998e-06 2024-09-22T01:01:08.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093176+0000", 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9590000000000001e-06 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093193+0000", 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6986999999999998e-05 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104890+0000", 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011696745 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104969+0000", 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9086999999999997e-05 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.105078+0000", 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108503 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:08.942 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:08.943 INFO:teuthology.orchestra.run.smithi002.stdout:1094 write 0x4151570 thru 0x415644f (0x4ee0 bytes) 2024-09-22T01:01:08.943 INFO:teuthology.orchestra.run.smithi002.stdout:1095 read 0x8f11508 thru 0x8f1ad52 (0x984b bytes) 2024-09-22T01:01:08.943 INFO:teuthology.orchestra.run.smithi002.stdout:1097 read 0xa568a02 thru 0xa56de2c (0x542b bytes) 2024-09-22T01:01:08.944 INFO:teuthology.orchestra.run.smithi002.stdout:1098 writesame 0xa9ea129 thru 0xa9ef2b9 data_size 0x13(0x5191 bytes) 2024-09-22T01:01:08.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:08.945 INFO:teuthology.orchestra.run.smithi002.stdout:1100 write 0xc95806d thru 0xc967169 (0xf0fd bytes) 2024-09-22T01:01:08.945 INFO:teuthology.orchestra.run.smithi002.stdout:1101 writesame 0x6a7561f thru 0x6a7b226 data_size 0x5f(0x5c08 bytes) 2024-09-22T01:01:08.945 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:1104 trunc from 0xc96716a to 0xe1c75e3 2024-09-22T01:01:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:1105 trunc from 0xe1c75e3 to 0xcf9255c 2024-09-22T01:01:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:1106 write 0xae3e366 thru 0xae3ef8b (0xc26 bytes) 2024-09-22T01:01:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:1107 punch from 0xa6033b5 to 0xa607218, (0x3e63 bytes) 2024-09-22T01:01:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:1108 write 0xed9464b thru 0xed9b0ac (0x6a62 bytes) 2024-09-22T01:01:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:1109 read 0xe5216dd thru 0xe52edc8 (0xd6ec bytes) 2024-09-22T01:01:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:1110 punch from 0x1c2100e to 0x1c2b3a5, (0xa397 bytes) 2024-09-22T01:01:08.946 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:08.945+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:1111 trunc from 0xed9b0ad to 0x650b479 2024-09-22T01:01:08.960 INFO:teuthology.orchestra.run.smithi002.stdout:1112 read 0x404d4c thru 0x412b31 (0xdde6 bytes) 2024-09-22T01:01:08.961 INFO:teuthology.orchestra.run.smithi002.stdout:1113 read 0x50d91c2 thru 0x50e0c61 (0x7aa0 bytes) 2024-09-22T01:01:08.962 INFO:teuthology.orchestra.run.smithi002.stdout:1114 write 0x36ec5de thru 0x36f748c (0xaeaf bytes) 2024-09-22T01:01:08.965 INFO:teuthology.orchestra.run.smithi002.stdout:1115 write 0x2e169c0 thru 0x2e17771 (0xdb2 bytes) 2024-09-22T01:01:08.967 INFO:teuthology.orchestra.run.smithi002.stdout:1117 write 0x48103a7 thru 0x4816858 (0x64b2 bytes) 2024-09-22T01:01:08.970 INFO:teuthology.orchestra.run.smithi002.stdout:1118 punch from 0x453d943 to 0x4546070, (0x872d bytes) 2024-09-22T01:01:08.970 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:08.970 INFO:teuthology.orchestra.run.smithi002.stdout:1121 read 0x5d2d5b7 thru 0x5d3bcef (0xe739 bytes) 2024-09-22T01:01:08.986 INFO:teuthology.orchestra.run.smithi002.stdout:1122 write 0xa69d8be thru 0xa69f892 (0x1fd5 bytes) 2024-09-22T01:01:08.989 INFO:teuthology.orchestra.run.smithi002.stdout:1123 write 0x49facd1 thru 0x4a0870f (0xda3f bytes) 2024-09-22T01:01:09.005 INFO:teuthology.orchestra.run.smithi002.stdout:1125 trunc from 0xa69f893 to 0x1c5e427 2024-09-22T01:01:09.022 INFO:teuthology.orchestra.run.smithi002.stdout:1126 clone 10 order 19 su 4096 sc 10 2024-09-22T01:01:09.045 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:09.044+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:09.082 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:09.082 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:09.082 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:09.082 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:09.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:09.185 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:09.184+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:09.246 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:09.245+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:09.268 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:09.268 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:09.269 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:09.269 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:09.269 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:09.284 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:09.369 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:09.368+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:09.448 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:09.444+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:09.454 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375247913999999, 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375229055, 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.37521087, 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375193081999999, 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375174524, 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375156472, 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375139123, 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375117169999999, 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375099635, 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375075849, 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375057987, 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.498 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375030750000001, 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:09.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:09.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.375012045, 2024-09-22T01:01:09.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:09.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.374988547999999, 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.374964579, 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:09.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.374945364, 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:09.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:09.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.373940814000001, 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:09.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:09.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.371487626, 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.531 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.371465546, 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:09.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:09.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:09.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.371434861000001, 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:09.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:09.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:09.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:09.540 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:09.541 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:09.584 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:09.585 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:09.584+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:09.651 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:09.649+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:09.709 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:09.709 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:09.709 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:09.709 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:09.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:09.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:09.747 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:09.752 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:09.753+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:09.765 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:09.765 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:09.783 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:09.783 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:09.800 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:09.800 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:09.821 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:09.821 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:09.822 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:09.844 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:09.844 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:09.849 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:09.849 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:09.852 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:09.848+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:09.861 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:09.872 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:09.873 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:09.877 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:09.877 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:09.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:01:09.965 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:09.964+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:09.999 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:09.999 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:10.017 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:10.031 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone9 2024-09-22T01:01:10.077 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:10.076+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:10.154 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:10.152+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:10.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-22T01:01:10.254 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:10.252+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:10.309 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:10.309 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:10.323 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:10.355 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:10.353+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:10.456 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:10.457+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:10.487 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:10.487 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:10.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:10.554 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:10.554 INFO:teuthology.orchestra.run.smithi002.stdout:1130 read 0x1ad4a42 thru 0x1adbd60 (0x731f bytes) 2024-09-22T01:01:10.556 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:10.557 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:10.557+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:10.575 INFO:teuthology.orchestra.run.smithi002.stdout:1132 write 0x8b3e468 thru 0x8b41ec2 (0x3a5b bytes) 2024-09-22T01:01:10.591 INFO:teuthology.orchestra.run.smithi002.stdout:1133 trunc from 0x8b41ec3 to 0xdb3a877 2024-09-22T01:01:10.599 INFO:teuthology.orchestra.run.smithi002.stdout:1136 read 0x8a6f9b2 thru 0x8a78d0b (0x935a bytes) 2024-09-22T01:01:10.600 INFO:teuthology.orchestra.run.smithi002.stdout:1137 write 0xdadc62a thru 0xdae7bfd (0xb5d4 bytes) 2024-09-22T01:01:10.614 INFO:teuthology.orchestra.run.smithi002.stdout:1138 write 0x161b82b thru 0x161df0a (0x26e0 bytes) 2024-09-22T01:01:10.645 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:10.645 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:10.647 INFO:teuthology.orchestra.run.smithi002.stdout:1140 clone 11 order 23 su 65536 sc 11 2024-09-22T01:01:10.657 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:10.656+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:10.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:10.758 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:10.757+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:10.822 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:10.822 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:10.836 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:10.853 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:01:10.880 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:10.881+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:10.960 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:10.956+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:10.969 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone10 from 0xdb3a877 (overlap 0x1c5e427) to 0x0 2024-09-22T01:01:11.030 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:11.030 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:11.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:11.099 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:11.101+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:11.162 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:11.160+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:11.229 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:11.229 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:11.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:11.262 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:11.260+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:11.364 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:11.360+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:11.433 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:11.433 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:11.447 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:11.447 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-22T01:01:11.496 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:11.497+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:11.515 INFO:teuthology.orchestra.run.smithi002.stdout:1141 write 0x651e546 thru 0x65257a0 (0x725b bytes) 2024-09-22T01:01:11.547 INFO:teuthology.orchestra.run.smithi002.stdout:1142 write 0x7ec52f9 thru 0x7ece10e (0x8e16 bytes) 2024-09-22T01:01:11.564 INFO:teuthology.orchestra.run.smithi002.stdout:1145 read 0x4f02201 thru 0x4f0c6b6 (0xa4b6 bytes) 2024-09-22T01:01:11.564 INFO:teuthology.orchestra.run.smithi002.stdout:1146 writesame 0x8355cd thru 0x839afe data_size 0x11(0x4532 bytes) 2024-09-22T01:01:11.565 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:11.565+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:11.573 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:11.573 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:11.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:11.666 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:11.664+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:11.669 INFO:teuthology.orchestra.run.smithi002.stdout:1147 trunc from 0xdb3a877 to 0x26d8565 2024-09-22T01:01:11.766 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:11.765+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:11.787 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:11.787 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:11.806 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:11.867 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:11.864+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:11.878 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:11.968 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:11.964+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:11.975 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:11.975 INFO:teuthology.orchestra.run.smithi002.stdout:1149 read 0x24f8aa1 thru 0x25081a5 (0xf705 bytes) 2024-09-22T01:01:11.991 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:11.991 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:11.995 INFO:teuthology.orchestra.run.smithi002.stdout:1150 write 0xbf2a0c5 thru 0xbf31d01 (0x7c3d bytes) 2024-09-22T01:01:11.999 INFO:teuthology.orchestra.run.smithi002.stdout:1153 punch from 0x136bb63 to 0x1378666, (0xcb03 bytes) 2024-09-22T01:01:12.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:12.007 INFO:teuthology.orchestra.run.smithi002.stdout:1154 write 0xc6596ea thru 0xc65a47a (0xd91 bytes) 2024-09-22T01:01:12.010 INFO:teuthology.orchestra.run.smithi002.stdout:1156 punch from 0x1862200 to 0x1870af7, (0xe8f7 bytes) 2024-09-22T01:01:12.010 INFO:teuthology.orchestra.run.smithi002.stdout:1157 write 0x5aa0789 thru 0x5aa683d (0x60b5 bytes) 2024-09-22T01:01:12.013 INFO:teuthology.orchestra.run.smithi002.stdout:1158 writesame 0x8cc5dcd thru 0x8ccd0ec data_size 0x20(0x7320 bytes) 2024-09-22T01:01:12.017 INFO:teuthology.orchestra.run.smithi002.stdout:1159 read 0x6fd7b18 thru 0x6fdc046 (0x452f bytes) 2024-09-22T01:01:12.017 INFO:teuthology.orchestra.run.smithi002.stdout:1161 read 0xb53a8ba thru 0xb549c2e (0xf375 bytes) 2024-09-22T01:01:12.017 INFO:teuthology.orchestra.run.smithi002.stdout:1163 read 0x7d9d3ea thru 0x7da4e4c (0x7a63 bytes) 2024-09-22T01:01:12.017 INFO:teuthology.orchestra.run.smithi002.stdout:1164 read 0xa19acae thru 0xa19c226 (0x1579 bytes) 2024-09-22T01:01:12.017 INFO:teuthology.orchestra.run.smithi002.stdout:1165 read 0xbf8ac52 thru 0xbf92af2 (0x7ea1 bytes) 2024-09-22T01:01:12.017 INFO:teuthology.orchestra.run.smithi002.stdout:1168 read 0xaea9706 thru 0xaeaca21 (0x331c bytes) 2024-09-22T01:01:12.022 INFO:teuthology.orchestra.run.smithi002.stdout:1169 trunc from 0xc65a47b to 0x8d1602c 2024-09-22T01:01:12.043 INFO:teuthology.orchestra.run.smithi002.stdout:1170 write 0xac79bdc thru 0xac7ad37 (0x115c bytes) 2024-09-22T01:01:12.044 INFO:teuthology.orchestra.run.smithi002.stdout:1171 read 0x1648042 thru 0x16523a1 (0xa360 bytes) 2024-09-22T01:01:12.045 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:12.045 INFO:teuthology.orchestra.run.smithi002.stdout:1173 punch from 0x295e7b3 to 0x2965874, (0x70c1 bytes) 2024-09-22T01:01:12.045 INFO:teuthology.orchestra.run.smithi002.stdout:1174 read 0x82e9a80 thru 0x82f57e2 (0xbd63 bytes) 2024-09-22T01:01:12.056 INFO:teuthology.orchestra.run.smithi002.stdout:1175 write 0xe761ca6 thru 0xe76ff76 (0xe2d1 bytes) 2024-09-22T01:01:12.058 INFO:teuthology.orchestra.run.smithi002.stdout:1176 writesame 0x82c0767 thru 0x82cd3b3 data_size 0x1(0xcc4d bytes) 2024-09-22T01:01:12.062 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:12.062 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:12.069 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:12.068+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:12.078 INFO:teuthology.orchestra.run.smithi002.stdout:1179 punch from 0x41a1cb to 0x423ec3, (0x9cf8 bytes) 2024-09-22T01:01:12.078 INFO:teuthology.orchestra.run.smithi002.stdout:1180 clone 12 order 25 su 1048576 sc 2 2024-09-22T01:01:12.079 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:12.079 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:12.096 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:12.113 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:12.113 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:12.123 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:12.123 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:12.133 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:12.133 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:12.133 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:12.137 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:12.153 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:12.153 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:12.170 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:12.168+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:12.174 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:12.174 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:12.177 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:12.178 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:12.263 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:12.263 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:12.263 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:12.263 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:12.272 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:12.273+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:12.277 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:12.372 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:12.373+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:12.403 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:12.403 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:12.403 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:12.404 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:12.404 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:12.418 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:12.473 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:12.472+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:12.540 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:12.540 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8692 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.449145576999999, 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01097323, 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8692 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089893+0000", 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089896+0000", 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089904+0000", 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3440000000000001e-06 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089911+0000", 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5799999999999997e-06 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100463+0000", 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010552713 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100732+0000", 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026876599999999998 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100866+0000", 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133623 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8693 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.449118537, 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011283726000000001, 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8693 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089920+0000", 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089924+0000", 2024-09-22T01:01:12.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089932+0000", 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9929999999999995e-06 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.089941+0000", 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9719999999999998e-06 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.100989+0000", 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011048231 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101092+0000", 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102975 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101203+0000", 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111648 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.549 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8712 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 630784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.449017573999999, 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011403979999999999, 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8712 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090021+0000", 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090022+0000", 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.19e-06 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090027+0000", 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.604e-06 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090041+0000", 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3114e-05 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090049+0000", 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6479999999999992e-06 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101235+0000", 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011185428000000001 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101309+0000", 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4376999999999996e-05 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101425+0000", 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011561900000000001 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8713 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.448973318, 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011573441, 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8713 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090065+0000", 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090069+0000", 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9840000000000003e-06 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090077+0000", 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6639999999999998e-06 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090083+0000", 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101474+0000", 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011391007 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101548+0000", 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4555999999999996e-05 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101638+0000", 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0223000000000007e-05 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8737 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.448947689000001, 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011746902, 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8737 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:12.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090091+0000", 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090093+0000", 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1619999999999998e-06 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090099+0000", 2024-09-22T01:01:12.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2249999999999997e-06 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090104+0000", 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9520000000000002e-06 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101668+0000", 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011563339000000001 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101743+0000", 2024-09-22T01:01:12.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5796000000000004e-05 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101837+0000", 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4055999999999995e-05 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8747 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 729088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.448925167000001, 2024-09-22T01:01:12.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011923398, 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8747 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090113+0000", 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9900000000000001e-07 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090127+0000", 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3447e-05 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090134+0000", 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3799999999999996e-06 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090139+0000", 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8119999999999998e-06 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101866+0000", 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011726752 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.101938+0000", 2024-09-22T01:01:12.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102036+0000", 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8041000000000005e-05 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8757 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.448890198999999, 2024-09-22T01:01:12.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012330969000000001, 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8757 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090148+0000", 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090151+0000", 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-22T01:01:12.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090159+0000", 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4869999999999996e-06 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090164+0000", 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3210000000000004e-06 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102065+0000", 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011901022000000001 2024-09-22T01:01:12.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102381+0000", 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031597199999999998 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102479+0000", 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8101000000000001e-05 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8767 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.448866551, 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012517782, 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8767 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090172+0000", 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090174+0000", 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090181+0000", 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6320000000000002e-06 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090186+0000", 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6949999999999999e-06 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102511+0000", 2024-09-22T01:01:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012325469 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102601+0000", 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9802000000000005e-05 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102689+0000", 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8571000000000003e-05 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8777 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.448841923, 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012693987, 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8777 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.575 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090196+0000", 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090197+0000", 2024-09-22T01:01:12.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090199+0000", 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090206+0000", 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4089999999999997e-06 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.090212+0000", 2024-09-22T01:01:12.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4380000000000003e-06 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102718+0000", 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012506735999999999 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102793+0000", 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5054000000000003e-05 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102890+0000", 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6825999999999998e-05 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8787 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 786432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.447563082, 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011612116, 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8787 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091475+0000", 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091476+0000", 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.11e-06 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091482+0000", 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091494+0000", 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2047e-05 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091503+0000", 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102918+0000", 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011414951 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.102992+0000", 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3279000000000001e-05 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103087+0000", 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5662000000000007e-05 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8797 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 802816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.447518925000001, 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011753412, 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8797 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091519+0000", 2024-09-22T01:01:12.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091520+0000", 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5000000000000003e-07 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091536+0000", 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6214000000000001e-05 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091546+0000", 2024-09-22T01:01:12.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7799999999999995e-06 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091559+0000", 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2911000000000001e-05 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103116+0000", 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011557207 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103188+0000", 2024-09-22T01:01:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2418999999999999e-05 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103273+0000", 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4331000000000003e-05 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8807 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 811008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.447466561000001, 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011888717, 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8807 2024-09-22T01:01:12.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.588 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091572+0000", 2024-09-22T01:01:12.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091576+0000", 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5540000000000002e-06 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091586+0000", 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0502999999999999e-05 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091593+0000", 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.861e-06 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103300+0000", 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011706467999999999 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103371+0000", 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1308000000000002e-05 2024-09-22T01:01:12.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103460+0000", 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9395999999999996e-05 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8818 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 761856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.447434649, 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012045223000000001, 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8818 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:12.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091604+0000", 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2200000000000004e-07 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091607+0000", 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5440000000000001e-06 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091615+0000", 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0169999999999996e-06 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091620+0000", 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-06 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103488+0000", 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011867353000000001 2024-09-22T01:01:12.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103561+0000", 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3481999999999999e-05 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103649+0000", 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7495000000000002e-05 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8821 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.447408576000001, 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012214868, 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8821 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091630+0000", 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3099999999999998e-07 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091633+0000", 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5770000000000001e-06 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091641+0000", 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091647+0000", 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1079999999999998e-06 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103676+0000", 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012029361000000001 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103747+0000", 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0587999999999996e-05 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103844+0000", 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7606e-05 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8822 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.447382274000001, 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01239211, 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8822 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.601 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091656+0000", 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091659+0000", 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5770000000000001e-06 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091667+0000", 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3410000000000003e-06 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091673+0000", 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8270000000000003e-06 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103872+0000", 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012199162 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.103951+0000", 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8512999999999999e-05 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104048+0000", 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7212000000000004e-05 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8824 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.447356516999999, 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012562043, 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.605 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8824 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091682+0000", 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091685+0000", 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5349999999999999e-06 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091693+0000", 2024-09-22T01:01:12.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3019999999999995e-06 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.091707+0000", 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4345000000000001e-05 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104080+0000", 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012372321 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104156+0000", 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6734000000000001e-05 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104244+0000", 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7363999999999999e-05 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8848 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.445990883, 2024-09-22T01:01:12.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011398933, 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8848 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.610 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093047+0000", 2024-09-22T01:01:12.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093048+0000", 2024-09-22T01:01:12.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.136e-06 2024-09-22T01:01:12.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093054+0000", 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6969999999999998e-06 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093074+0000", 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.95e-05 2024-09-22T01:01:12.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093083+0000", 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3479999999999993e-06 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104273+0000", 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011189747999999999 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104352+0000", 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9560000000000004e-05 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104446+0000", 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3943999999999994e-05 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8849 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 892928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.445938528999999, 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011546511000000001, 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8849 2024-09-22T01:01:12.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093100+0000", 2024-09-22T01:01:12.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:01:12.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093104+0000", 2024-09-22T01:01:12.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4010000000000001e-06 2024-09-22T01:01:12.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093113+0000", 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1349999999999998e-06 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093119+0000", 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3720000000000001e-06 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104477+0000", 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011358157000000001 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104554+0000", 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6390000000000006e-05 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104646+0000", 2024-09-22T01:01:12.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2379000000000004e-05 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8868 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 860160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.445908382000001, 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011726966, 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8868 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:12.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093130+0000", 2024-09-22T01:01:12.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5999999999999999e-07 2024-09-22T01:01:12.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093133+0000", 2024-09-22T01:01:12.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6900000000000001e-06 2024-09-22T01:01:12.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093142+0000", 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6780000000000006e-06 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093148+0000", 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1160000000000004e-06 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104677+0000", 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011529285 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104756+0000", 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104857+0000", 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010123099999999999 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:8869 3.1 3:85a1ea75:::rbd_data.117c1b47bdd2.0000000000000001:4 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.44587508, 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011914377, 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8869 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093163+0000", 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:12.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093164+0000", 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093167+0000", 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4359999999999998e-06 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093176+0000", 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9590000000000001e-06 2024-09-22T01:01:12.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.093193+0000", 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6986999999999998e-05 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104890+0000", 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011696745 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:12.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.104969+0000", 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9086999999999997e-05 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.105078+0000", 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108503 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:12.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:12.633 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:12.632+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:12.734 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:12.732+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:12.809 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:12.809 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:12.809 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:12.809 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:12.823 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:12.865 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:12.864+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:12.885 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:01:12.935 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:12.937+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:12.957 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone11 from 0xe76ff77 (overlap 0x26d8565) to 0xba74d1 2024-09-22T01:01:13.003 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:13.003 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:13.004 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:13.004 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:13.004 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:13.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:13.036 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:13.037+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:13.071 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-22T01:01:13.072 INFO:teuthology.orchestra.run.smithi002.stdout:1181 punch from 0xe136eb to 0xe1949e, (0x5db3 bytes) 2024-09-22T01:01:13.077 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.077 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.078 INFO:teuthology.orchestra.run.smithi002.stdout:1189 read 0x452458 thru 0x45e488 (0xc031 bytes) 2024-09-22T01:01:13.078 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.078 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.079 INFO:teuthology.orchestra.run.smithi002.stdout:1193 punch from 0x466fd99 to 0x467a318, (0xa57f bytes) 2024-09-22T01:01:13.079 INFO:teuthology.orchestra.run.smithi002.stdout:1194 write 0xc9a5c34 thru 0xc9b3110 (0xd4dd bytes) 2024-09-22T01:01:13.092 INFO:teuthology.orchestra.run.smithi002.stdout:1197 write 0x48ea0f6 thru 0x48f1037 (0x6f42 bytes) 2024-09-22T01:01:13.129 INFO:teuthology.orchestra.run.smithi002.stdout:1201 read 0xf60225 thru 0xf675d6 (0x73b2 bytes) 2024-09-22T01:01:13.130 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.130 INFO:teuthology.orchestra.run.smithi002.stdout:1203 writesame 0xbb8bae2 thru 0xbb9308b data_size 0x1(0x75aa bytes) 2024-09-22T01:01:13.136 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:13.132+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:13.155 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.076147072000001, 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.076128213, 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.076110027999999, 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.076092240000001, 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.076073682000001, 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.076055629999999, 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.076038280999999, 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.076016328000001, 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.075998793, 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.075975007, 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.075957145, 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.075929907999999, 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.075911203, 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.075887706, 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.075863736999999, 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.075844522000001, 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.074839971999999, 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.072386783999999, 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:13.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:13.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.072364704000002, 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:13.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.072334018999999, 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:13.233 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.237 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:13.238 INFO:teuthology.orchestra.run.smithi002.stdout:1205 read 0x26f46e7 thru 0x26f5fa1 (0x18bb bytes) 2024-09-22T01:01:13.238 INFO:teuthology.orchestra.run.smithi002.stdout:1206 trunc from 0xe76ff77 to 0xbe3c773 2024-09-22T01:01:13.239 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:13.239 INFO:teuthology.orchestra.run.smithi002.stdout:1207 punch from 0x23afe35 to 0x23b0bc7, (0xd92 bytes) 2024-09-22T01:01:13.240 INFO:teuthology.orchestra.run.smithi002.stdout:1208 write 0x8328769 thru 0x8334c11 (0xc4a9 bytes) 2024-09-22T01:01:13.240 INFO:teuthology.orchestra.run.smithi002.stdout:1209 read 0x9df365e thru 0x9e029e4 (0xf387 bytes) 2024-09-22T01:01:13.240 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.240 INFO:teuthology.orchestra.run.smithi002.stdout:1211 read 0x5fe9173 thru 0x5feaa43 (0x18d1 bytes) 2024-09-22T01:01:13.240 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.240 INFO:teuthology.orchestra.run.smithi002.stdout:1213 writesame 0x5321fcd thru 0x5327311 data_size 0x1(0x5345 bytes) 2024-09-22T01:01:13.240 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:13.237+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:13.264 INFO:teuthology.orchestra.run.smithi002.stdout:1215 write 0x528cb78 thru 0x529ab0a (0xdf93 bytes) 2024-09-22T01:01:13.267 INFO:teuthology.orchestra.run.smithi002.stdout:1216 read 0xadb6f0e thru 0xadbc2e0 (0x53d3 bytes) 2024-09-22T01:01:13.268 INFO:teuthology.orchestra.run.smithi002.stdout:1219 write 0x2dad891 thru 0x2daf621 (0x1d91 bytes) 2024-09-22T01:01:13.340 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:13.341+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:13.388 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:13.388 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:13.389 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:13.389 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:13.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:13.445 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:13.444+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:13.530 INFO:teuthology.orchestra.run.smithi002.stdout:1220 read 0x93b6e96 thru 0x93b709e (0x209 bytes) 2024-09-22T01:01:13.531 INFO:teuthology.orchestra.run.smithi002.stdout:1221 read 0x1ec1765 thru 0x1ec571f (0x3fbb bytes) 2024-09-22T01:01:13.532 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.532 INFO:teuthology.orchestra.run.smithi002.stdout:1225 read 0x26df6b7 thru 0x26ece56 (0xd7a0 bytes) 2024-09-22T01:01:13.533 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.541 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:13.540+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:13.550 INFO:teuthology.orchestra.run.smithi002.stdout:1230 write 0xd624849 thru 0xd629d59 (0x5511 bytes) 2024-09-22T01:01:13.558 INFO:teuthology.orchestra.run.smithi002.stdout:1231 read 0x20a7ef3 thru 0x20b2029 (0xa137 bytes) 2024-09-22T01:01:13.559 INFO:teuthology.orchestra.run.smithi002.stdout:1233 read 0x29eed thru 0x2ed49 (0x4e5d bytes) 2024-09-22T01:01:13.560 INFO:teuthology.orchestra.run.smithi002.stdout:1234 writesame 0x5463d46 thru 0x54655a7 data_size 0x1(0x1862 bytes) 2024-09-22T01:01:13.569 INFO:teuthology.orchestra.run.smithi002.stdout:1235 write 0xc337769 thru 0xc3438ef (0xc187 bytes) 2024-09-22T01:01:13.575 INFO:teuthology.orchestra.run.smithi002.stdout:1236 read 0x48ee99e thru 0x48f9856 (0xaeb9 bytes) 2024-09-22T01:01:13.577 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.577 INFO:teuthology.orchestra.run.smithi002.stdout:1238 read 0x14bc0f7 thru 0x14c5be7 (0x9af1 bytes) 2024-09-22T01:01:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:1242 punch from 0xaa06a79 to 0xaa15914, (0xee9b bytes) 2024-09-22T01:01:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:1244 read 0x55ad80e thru 0x55b3d9f (0x6592 bytes) 2024-09-22T01:01:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:1246 read 0x2fdce69 thru 0x2fe6ce7 (0x9e7f bytes) 2024-09-22T01:01:13.581 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:13.581 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:13.600 INFO:teuthology.orchestra.run.smithi002.stdout:1248 trunc from 0xd629d5a to 0x4895769 2024-09-22T01:01:13.603 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:13.624 INFO:teuthology.orchestra.run.smithi002.stdout:1249 punch from 0x10397fb to 0x103ab78, (0x137d bytes) 2024-09-22T01:01:13.624 INFO:teuthology.orchestra.run.smithi002.stdout:1250 read 0x23d7398 thru 0x23e6273 (0xeedc bytes) 2024-09-22T01:01:13.642 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:13.641+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:13.643 INFO:teuthology.orchestra.run.smithi002.stdout:1252 trunc from 0x4895769 to 0xc18d217 2024-09-22T01:01:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:1253 write 0x1a78271 thru 0x1a82d35 (0xaac5 bytes) 2024-09-22T01:01:13.743 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:13.741+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:13.763 INFO:teuthology.orchestra.run.smithi002.stdout:1254 write 0x23fd180 thru 0x240158a (0x440b bytes) 2024-09-22T01:01:13.764 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4235.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.832776166000002, 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015255189999999999, 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930032+0000", 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-07 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930049+0000", 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7669999999999999e-05 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930072+0000", 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2232e-05 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930090+0000", 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8491999999999998e-05 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930185+0000", 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4500999999999995e-05 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930236+0000", 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1798000000000001e-05 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.931471+0000", 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001234228 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.931557+0000", 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5853999999999995e-05 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:1943 3.0 e28/22 3:046f320d:::rbd_data.117c1b47bdd2.000000000000003b:head v 28'6, mlcod=25'3)", 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.632690179000001, 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017318469999999999, 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130118+0000", 2024-09-22T01:01:13.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3900000000000004e-07 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130185+0000", 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7076999999999995e-05 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130201+0000", 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6320999999999998e-05 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130210+0000", 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9279999999999999e-06 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130349+0000", 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013925799999999999 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130372+0000", 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2660999999999999e-05 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.131801+0000", 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001429034 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.131849+0000", 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7729000000000003e-05 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:7990 3.4 e32/22 3:2775b4d4:::rbd_data.117ce1c46ae.0000000000000032:head v 32'16, mlcod=31'12)", 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:58.941629+0000", 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.821177701, 2024-09-22T01:01:13.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014661279999999999, 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941629+0000", 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941629+0000", 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941631+0000", 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3850000000000001e-06 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941641+0000", 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9559999999999992e-06 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941667+0000", 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5965999999999998e-05 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941677+0000", 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0234999999999999e-05 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941743+0000", 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6192000000000006e-05 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941766+0000", 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2429000000000002e-05 2024-09-22T01:01:13.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.943039+0000", 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012731070000000001 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.943096+0000", 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6857999999999997e-05 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8071 3.4 e32/22 3:230ffd57:::rbd_data.117ce1c46ae.0000000000000018:head v 32'17, mlcod=32'14)", 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:58.959772+0000", 2024-09-22T01:01:13.777 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.803034825999999, 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017058590000000001, 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959772+0000", 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959772+0000", 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959773+0000", 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5000000000000001e-07 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959955+0000", 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018200399999999999 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959969+0000", 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3857e-05 2024-09-22T01:01:13.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959978+0000", 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9409999999999996e-06 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.960031+0000", 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3402000000000001e-05 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.960047+0000", 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5231e-05 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.961427+0000", 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001380559 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.961478+0000", 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1014999999999999e-05 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8964 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'20, mlcod=32'17)", 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:13.781 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.626283852, 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077894110000000004, 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:13.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136525+0000", 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3540000000000001e-06 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137650+0000", 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011257190000000001 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137671+0000", 2024-09-22T01:01:13.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0560999999999998e-05 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137680+0000", 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137763+0000", 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2472999999999995e-05 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137793+0000", 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-05 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144267+0000", 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064739139999999999 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144313+0000", 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5624999999999998e-05 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8968 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'21, mlcod=32'17)", 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.621857807, 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004894771, 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:13.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140952+0000", 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1849999999999998e-06 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.141985+0000", 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001032984 2024-09-22T01:01:13.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142001+0000", 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6776999999999999e-05 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142010+0000", 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4640000000000006e-06 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142101+0000", 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1312999999999998e-05 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142119+0000", 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8202e-05 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145810+0000", 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036909970000000001 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145844+0000", 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3849000000000001e-05 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8991 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'22, mlcod=32'20)", 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.231258+0000", 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.531549439999999, 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017482730000000001, 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231258+0000", 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231258+0000", 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231260+0000", 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4439999999999998e-06 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231305+0000", 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5118999999999999e-05 2024-09-22T01:01:13.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231341+0000", 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5500000000000002e-05 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231362+0000", 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0696e-05 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231527+0000", 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165043 2024-09-22T01:01:13.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231587+0000", 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0862000000000001e-05 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.232906+0000", 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00131909 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.233006+0000", 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9518999999999994e-05 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(3.4 epoch 32/22 rep_tid 55 entries 32'26 (0'0) error 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head by client.4476.0:9032 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 32'24)", 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.304780+0000", 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.458027458, 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002035411, 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304780+0000", 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304780+0000", 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304783+0000", 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304809+0000", 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6207000000000002e-05 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304875+0000", 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6415999999999996e-05 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304917+0000", 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1539999999999999e-05 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305262+0000", 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00034516000000000001 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.306815+0000", 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015529680000000001 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(3.4 epoch 32/22 rep_tid 56 entries 32'27 (0'0) error 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head by client.4476.0:9033 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 32'24)", 2024-09-22T01:01:13.797 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.305934+0000", 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.456873243, 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016401110000000001, 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305934+0000", 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305934+0000", 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305936+0000", 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8059999999999999e-06 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305956+0000", 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9993999999999999e-05 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305999+0000", 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3495999999999998e-05 2024-09-22T01:01:13.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.306035+0000", 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5565999999999998e-05 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.306266+0000", 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000231435 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.307574+0000", 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001307814 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9048 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'28, mlcod=32'26)", 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.320562+0000", 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.442245505000001, 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019977699999999998, 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320562+0000", 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320562+0000", 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320564+0000", 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.934e-06 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320627+0000", 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3546000000000004e-05 2024-09-22T01:01:13.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320654+0000", 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7117000000000001e-05 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320670+0000", 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5311999999999998e-05 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320763+0000", 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3202000000000007e-05 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320798+0000", 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5398999999999998e-05 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.322477+0000", 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016784739999999999 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.322559+0000", 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2786000000000003e-05 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9061 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'30, mlcod=32'28)", 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.337997+0000", 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.42480978, 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001560664, 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.337997+0000", 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.337997+0000", 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.337999+0000", 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.736e-06 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338078+0000", 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8652999999999994e-05 2024-09-22T01:01:13.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338111+0000", 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2863e-05 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338128+0000", 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7017e-05 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338225+0000", 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7083000000000005e-05 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338259+0000", 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4400000000000003e-05 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.339432+0000", 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00117303 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.339558+0000", 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012588200000000001 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9077 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'31, mlcod=32'29)", 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.367533432, 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034796760000000001, 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395277+0000", 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8600000000000001e-06 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395294+0000", 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7513e-05 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395337+0000", 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2749e-05 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395358+0000", 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1097000000000001e-05 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395548+0000", 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189829 2024-09-22T01:01:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395593+0000", 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4919000000000001e-05 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398634+0000", 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003041373 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398753+0000", 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119336 2024-09-22T01:01:13.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9083 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'32, mlcod=32'30)", 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.361752150999999, 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002882763, 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:13.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401057+0000", 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.677e-06 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401069+0000", 2024-09-22T01:01:13.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2673e-05 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401107+0000", 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7200000000000003e-05 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401122+0000", 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5203e-05 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401251+0000", 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012935900000000001 2024-09-22T01:01:13.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401287+0000", 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5633000000000003e-05 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.403853+0000", 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025658389999999999 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.403938+0000", 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5179000000000006e-05 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.816 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9847 3.0 e33/22 3:063de76a:::rbd_data.117ccb654ee9.0000000000000012:head v 33'23, mlcod=31'20)", 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.301313+0000", 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.461493691999999, 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001906673, 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301313+0000", 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301313+0000", 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301314+0000", 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6299999999999993e-07 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301366+0000", 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1748e-05 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301390+0000", 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3796999999999999e-05 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301403+0000", 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3200000000000001e-05 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301531+0000", 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000128045 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301570+0000", 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8856999999999997e-05 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.303099+0000", 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001528611 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.303220+0000", 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000121452 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9855 3.0 e33/22 3:063de76a:::rbd_data.117ccb654ee9.0000000000000012:head v 33'25, mlcod=33'23)", 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.337905+0000", 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.42490226, 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018437970000000001, 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337905+0000", 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337905+0000", 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337907+0000", 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0999999999999998e-06 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337962+0000", 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5186999999999999e-05 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337995+0000", 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3015999999999997e-05 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.338020+0000", 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5103e-05 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.338186+0000", 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016595600000000001 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.338255+0000", 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8425999999999998e-05 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.339667+0000", 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014121959999999999 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.339749+0000", 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1812999999999997e-05 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9864 3.0 e33/22 3:063de76a:::rbd_data.117ccb654ee9.0000000000000012:head v 33'27, mlcod=33'25)", 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.365897+0000", 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.396910452, 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001575611, 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365897+0000", 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365897+0000", 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365899+0000", 2024-09-22T01:01:13.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8560000000000001e-06 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365914+0000", 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5630000000000001e-05 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365956+0000", 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1486999999999998e-05 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365975+0000", 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9383000000000001e-05 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.366175+0000", 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020039799999999999 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.366230+0000", 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4429999999999999e-05 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.367385+0000", 2024-09-22T01:01:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011547739999999999 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.367472+0000", 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7652999999999996e-05 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9941 3.0 e33/22 3:063de76a:::rbd_data.117ccb654ee9.0000000000000012:head v 33'29, mlcod=33'27)", 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.484122+0000", 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.278684768, 2024-09-22T01:01:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018964240000000001, 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484122+0000", 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484122+0000", 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484125+0000", 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5660000000000001e-06 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484144+0000", 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9029999999999999e-05 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484186+0000", 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1489999999999997e-05 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484206+0000", 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0956999999999999e-05 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484424+0000", 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021716800000000001 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484478+0000", 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4725999999999997e-05 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.485960+0000", 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014821439999999999 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.486019+0000", 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8343e-05 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9958 3.9 e33/22 3:91af50a5:::rbd_data.117ccb654ee9.0000000000000002:head v 33'15, mlcod=31'13)", 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.497892+0000", 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.264915076999999, 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015364160000000001, 2024-09-22T01:01:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497892+0000", 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497892+0000", 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497894+0000", 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6530000000000001e-06 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497918+0000", 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3862999999999998e-05 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497947+0000", 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9442999999999999e-05 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497961+0000", 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.376e-05 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.498039+0000", 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8659999999999996e-05 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.498063+0000", 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3591000000000002e-05 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.499380+0000", 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001316953 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.499429+0000", 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8492999999999998e-05 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:14402 3.4 e39/22 3:25583b41:::rbd_data.117c1bff5a72.000000000000000d:head v 39'61, mlcod=39'59)", 2024-09-22T01:01:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:08.933151+0000", 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8296561440000003, 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015489220000000001, 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933151+0000", 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933151+0000", 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933153+0000", 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.81e-06 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933188+0000", 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4903000000000001e-05 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933209+0000", 2024-09-22T01:01:13.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1214999999999999e-05 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933218+0000", 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4460000000000007e-06 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933290+0000", 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1440999999999996e-05 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933317+0000", 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7053999999999999e-05 2024-09-22T01:01:13.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.934609+0000", 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012921059999999999 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.934700+0000", 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0946999999999995e-05 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:14416 3.4 e39/22 3:25583b41:::rbd_data.117c1bff5a72.000000000000000d:head v 39'62, mlcod=39'60)", 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:08.950669+0000", 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8121384670000005, 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001939493, 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950669+0000", 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950669+0000", 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950670+0000", 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7129999999999999e-06 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950685+0000", 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4460999999999999e-05 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950716+0000", 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1028999999999997e-05 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950729+0000", 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2787e-05 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950869+0000", 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000139989 2024-09-22T01:01:13.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950935+0000", 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6767999999999995e-05 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.952563+0000", 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016272190000000001 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.952608+0000", 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5527e-05 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:13.846 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:13.846 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-22T01:01:13.846 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:01:13.846 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-22T01:01:13.846 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:01:13.846 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:01:13.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-22T01:01:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:1257 read 0xbede39 thru 0xbf78ab (0x9a73 bytes) 2024-09-22T01:01:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:1261 punch from 0xac96e64 to 0xac9d696, (0x6832 bytes) 2024-09-22T01:01:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.849 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:1263 trunc from 0xc18d217 to 0x476315a 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:1264 trunc from 0x476315a to 0x99dfb41 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:1266 read 0x302bf6b thru 0x3039d1e (0xddb4 bytes) 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:1268 write 0x51a365a thru 0x51a450e (0xeb5 bytes) 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:1269 punch from 0x327737b to 0x3286072, (0xecf7 bytes) 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:1270 writesame 0x9034206 thru 0x904121f data_size 0x6(0xd01a bytes) 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:1271 read 0x4b4e43c thru 0x4b56d84 (0x8949 bytes) 2024-09-22T01:01:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:1272 read 0x973a3cb thru 0x9748fec (0xec22 bytes) 2024-09-22T01:01:13.851 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:13.849+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:1274 trunc from 0x99dfb41 to 0xc064af7 2024-09-22T01:01:13.865 INFO:teuthology.orchestra.run.smithi002.stdout:1275 punch from 0x4dcdb18 to 0x4ddcc23, (0xf10b bytes) 2024-09-22T01:01:13.865 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.865 INFO:teuthology.orchestra.run.smithi002.stdout:1277 read 0x15e9171 thru 0x15ecfa0 (0x3e30 bytes) 2024-09-22T01:01:13.866 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:01:13.866 INFO:teuthology.orchestra.run.smithi002.stdout:1278 write 0x1882cbe thru 0x18921ac (0xf4ef bytes) 2024-09-22T01:01:13.875 INFO:teuthology.orchestra.run.smithi002.stdout:1279 read 0x8f3eb34 thru 0x8f3faa1 (0xf6e bytes) 2024-09-22T01:01:13.875 INFO:teuthology.orchestra.run.smithi002.stdout:1280 write 0xbb63598 thru 0xbb6c6b9 (0x9122 bytes) 2024-09-22T01:01:13.881 INFO:teuthology.orchestra.run.smithi002.stdout:1281 read 0x8111265 thru 0x811720b (0x5fa7 bytes) 2024-09-22T01:01:13.881 INFO:teuthology.orchestra.run.smithi002.stdout:1282 read 0x1966411 thru 0x196f66e (0x925e bytes) 2024-09-22T01:01:13.882 INFO:teuthology.orchestra.run.smithi002.stdout:1283 write 0x1f4f511 thru 0x1f5c1f2 (0xcce2 bytes) 2024-09-22T01:01:13.885 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.896 INFO:teuthology.orchestra.run.smithi002.stdout:1285 write 0xeb6a62f thru 0xeb6d1bd (0x2b8f bytes) 2024-09-22T01:01:13.910 INFO:teuthology.orchestra.run.smithi002.stdout:1286 trunc from 0xeb6d1be to 0x81a8958 2024-09-22T01:01:13.925 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.925 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:13.934 INFO:teuthology.orchestra.run.smithi002.stdout:1289 writesame 0x894e421 thru 0x8958305 data_size 0x5b(0x9ee5 bytes) 2024-09-22T01:01:13.943 INFO:teuthology.orchestra.run.smithi002.stdout:1290 writesame 0x95ad8db thru 0x95b950b data_size 0x65(0xbc31 bytes) 2024-09-22T01:01:13.944 INFO:teuthology.orchestra.run.smithi002.stdout:1292 punch from 0x184f4cb to 0x18591a0, (0x9cd5 bytes) 2024-09-22T01:01:13.944 INFO:teuthology.orchestra.run.smithi002.stdout:1294 read 0x821634e thru 0x821ad9f (0x4a52 bytes) 2024-09-22T01:01:13.948 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:13.949+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:13.950 INFO:teuthology.orchestra.run.smithi002.stdout:1295 trunc from 0x95b950c to 0xd4287d9 2024-09-22T01:01:13.965 INFO:teuthology.orchestra.run.smithi002.stdout:1296 trunc from 0xd4287d9 to 0x7db99a9 2024-09-22T01:01:13.982 INFO:teuthology.orchestra.run.smithi002.stdout:1297 punch from 0x19a39fa to 0x19ad82d, (0x9e33 bytes) 2024-09-22T01:01:13.982 INFO:teuthology.orchestra.run.smithi002.stdout:1298 write 0x69b0417 thru 0x69b5995 (0x557f bytes) 2024-09-22T01:01:13.985 INFO:teuthology.orchestra.run.smithi002.stdout:1299 read 0x36216b0 thru 0x3624288 (0x2bd9 bytes) 2024-09-22T01:01:13.986 INFO:teuthology.orchestra.run.smithi002.stdout:1300 write 0x3838a09 thru 0x384190f (0x8f07 bytes) 2024-09-22T01:01:13.989 INFO:teuthology.orchestra.run.smithi002.stdout:1301 write 0x141bb10 thru 0x141e1ce (0x26bf bytes) 2024-09-22T01:01:13.992 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:13.992 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-22T01:01:13.992 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:13.992 INFO:teuthology.orchestra.run.smithi002.stdout:1304 read 0x407aff9 thru 0x40819e7 (0x69ef bytes) 2024-09-22T01:01:13.992 INFO:teuthology.orchestra.run.smithi002.stdout:1305 write 0x462d2bc thru 0x4630685 (0x33ca bytes) 2024-09-22T01:01:14.002 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:14.002 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:14.002 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:14.002 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:14.003 INFO:teuthology.orchestra.run.smithi002.stdout:1306 trunc from 0x7db99a9 to 0x2cfbc20 2024-09-22T01:01:14.018 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:14.042 INFO:teuthology.orchestra.run.smithi002.stdout:1308 write 0xb4a3215 thru 0xb4aebd9 (0xb9c5 bytes) 2024-09-22T01:01:14.047 INFO:teuthology.orchestra.run.smithi002.stdout:1309 write 0x96d419e thru 0x96d4a08 (0x86b bytes) 2024-09-22T01:01:14.048 INFO:teuthology.orchestra.run.smithi002.stdout:1310 read 0x7253153 thru 0x725ed8c (0xbc3a bytes) 2024-09-22T01:01:14.048 INFO:teuthology.orchestra.run.smithi002.stdout:1311 read 0x1870165 thru 0x187b3ff (0xb29b bytes) 2024-09-22T01:01:14.049 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:14.048+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:14.049 INFO:teuthology.orchestra.run.smithi002.stdout:1312 write 0x2dfce21 thru 0x2e03f43 (0x7123 bytes) 2024-09-22T01:01:14.055 INFO:teuthology.orchestra.run.smithi002.stdout:1313 read 0x5345505 thru 0x534ca40 (0x753c bytes) 2024-09-22T01:01:14.055 INFO:teuthology.orchestra.run.smithi002.stdout:1314 read 0x2f0ea87 thru 0x2f14203 (0x577d bytes) 2024-09-22T01:01:14.055 INFO:teuthology.orchestra.run.smithi002.stdout:1317 write 0xaa0e6d2 thru 0xaa10671 (0x1fa0 bytes) 2024-09-22T01:01:14.060 INFO:teuthology.orchestra.run.smithi002.stdout:1320 trunc from 0xb4aebda to 0x928aa0a 2024-09-22T01:01:14.069 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:14.069 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:14.069 INFO:teuthology.orchestra.run.smithi002.stdout:1323 writesame 0x6427517 thru 0x642c452 data_size 0x16(0x4f3c bytes) 2024-09-22T01:01:14.075 INFO:teuthology.orchestra.run.smithi002.stdout:1324 read 0x5c3faa3 thru 0x5c40f2c (0x148a bytes) 2024-09-22T01:01:14.076 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:14.077 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:14.077 INFO:teuthology.orchestra.run.smithi002.stdout:1327 punch from 0x2b49000 to 0x2b4b9a7, (0x29a7 bytes) 2024-09-22T01:01:14.077 INFO:teuthology.orchestra.run.smithi002.stdout:1329 read 0x1693077 thru 0x169f14f (0xc0d9 bytes) 2024-09-22T01:01:14.078 INFO:teuthology.orchestra.run.smithi002.stdout:1330 read 0x2d5bfff thru 0x2d65887 (0x9889 bytes) 2024-09-22T01:01:14.079 INFO:teuthology.orchestra.run.smithi002.stdout:1331 write 0x8ce9955 thru 0x8cf0ee4 (0x7590 bytes) 2024-09-22T01:01:14.089 INFO:teuthology.orchestra.run.smithi002.stdout:1332 trunc from 0x928aa0a to 0xe5695ee 2024-09-22T01:01:14.097 INFO:teuthology.orchestra.run.smithi002.stdout:1333 read 0xcdf9d2e thru 0xce020f9 (0x83cc bytes) 2024-09-22T01:01:14.097 INFO:teuthology.orchestra.run.smithi002.stdout:1335 read 0x67accc2 thru 0x67b3cb5 (0x6ff4 bytes) 2024-09-22T01:01:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:1336 trunc from 0xe5695ee to 0x3aa09dd 2024-09-22T01:01:14.140 INFO:teuthology.orchestra.run.smithi002.stdout:1338 trunc from 0x3aa09dd to 0x7de8bae 2024-09-22T01:01:14.148 INFO:teuthology.orchestra.run.smithi002.stdout:1339 write 0x55d9a33 thru 0x55de5b4 (0x4b82 bytes) 2024-09-22T01:01:14.149 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:14.148+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:14.165 INFO:teuthology.orchestra.run.smithi002.stdout:1340 trunc from 0x7de8bae to 0xe30eff5 2024-09-22T01:01:14.174 INFO:teuthology.orchestra.run.smithi002.stdout:1341 writesame 0xb52fb9d thru 0xb53b5aa data_size 0x37(0xba0e bytes) 2024-09-22T01:01:14.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:14.181 INFO:teuthology.orchestra.run.smithi002.stdout:1342 write 0x24626a5 thru 0x24659ea (0x3346 bytes) 2024-09-22T01:01:14.184 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:14.187 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:14.188 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:14.196 INFO:teuthology.orchestra.run.smithi002.stdout:1344 trunc from 0xe30eff5 to 0x6e7092b 2024-09-22T01:01:14.208 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:14.218 INFO:teuthology.orchestra.run.smithi002.stdout:1345 read 0x2fd1fa4 thru 0x2fdefb1 (0xd00e bytes) 2024-09-22T01:01:14.219 INFO:teuthology.orchestra.run.smithi002.stdout:1346 read 0x57858be thru 0x578b5b0 (0x5cf3 bytes) 2024-09-22T01:01:14.220 INFO:teuthology.orchestra.run.smithi002.stdout:1348 write 0x4e9e78c thru 0x4ea0b7c (0x23f1 bytes) 2024-09-22T01:01:14.226 INFO:teuthology.orchestra.run.smithi002.stdout:1349 read 0x4f4713f thru 0x4f4ec9b (0x7b5d bytes) 2024-09-22T01:01:14.227 INFO:teuthology.orchestra.run.smithi002.stdout:1350 read 0x5552399 thru 0x555d60a (0xb272 bytes) 2024-09-22T01:01:14.228 INFO:teuthology.orchestra.run.smithi002.stdout:1351 punch from 0x10ffc1f to 0x1103471, (0x3852 bytes) 2024-09-22T01:01:14.228 INFO:teuthology.orchestra.run.smithi002.stdout:1352 write 0x1487284 thru 0x1490f82 (0x9cff bytes) 2024-09-22T01:01:14.232 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:14.232 INFO:teuthology.orchestra.run.smithi002.stdout:1355 punch from 0x268c88f to 0x268e02c, (0x179d bytes) 2024-09-22T01:01:14.235 INFO:teuthology.orchestra.run.smithi002.stdout:1356 trunc from 0x6e7092b to 0x50d370c 2024-09-22T01:01:14.247 INFO:teuthology.orchestra.run.smithi002.stdout:1357 read 0x35df54c thru 0x35ed84b (0xe300 bytes) 2024-09-22T01:01:14.248 INFO:teuthology.orchestra.run.smithi002.stdout:1358 writesame 0x1bfabb2 thru 0x1c08bd2 data_size 0x1(0xe021 bytes) 2024-09-22T01:01:14.249 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:14.248+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:14.318 INFO:teuthology.orchestra.run.smithi002.stdout:1360 write 0xd3278b4 thru 0xd32c97d (0x50ca bytes) 2024-09-22T01:01:14.322 INFO:teuthology.orchestra.run.smithi002.stdout:1361 writesame 0x2523e44 thru 0x2533db2 data_size 0x3(0xff6f bytes) 2024-09-22T01:01:14.338 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:14.338 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:14.343 INFO:teuthology.orchestra.run.smithi002.stdout:1362 read 0x8bc4d6a thru 0x8bd0e1b (0xc0b2 bytes) 2024-09-22T01:01:14.343 INFO:teuthology.orchestra.run.smithi002.stdout:1363 read 0x87df5ff thru 0x87e87bb (0x91bd bytes) 2024-09-22T01:01:14.343 INFO:teuthology.orchestra.run.smithi002.stdout:1365 write 0x83be911 thru 0x83c23ef (0x3adf bytes) 2024-09-22T01:01:14.350 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:14.348+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1182 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.290428667, 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054296337, 2024-09-22T01:01:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1182 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2399999999999999e-07 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061556+0000", 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3700000000000002e-06 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061563+0000", 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0149999999999998e-06 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061568+0000", 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115682+0000", 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054113513000000002 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115767+0000", 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4670000000000001e-05 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115850+0000", 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3264999999999996e-05 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1183 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.290406931, 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054497446999999997, 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1183 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061576+0000", 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.65e-07 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061585+0000", 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4960000000000002e-06 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061593+0000", 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5340000000000002e-06 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061603+0000", 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0213e-05 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115905+0000", 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054302187000000002 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115985+0000", 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9429000000000001e-05 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116073+0000", 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8222999999999999e-05 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1184 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.290371602, 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054590382999999999, 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1184 2024-09-22T01:01:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061613+0000", 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061620+0000", 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.781e-06 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061625+0000", 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8999999999999997e-06 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116113+0000", 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054488414999999998 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116154+0000", 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0552000000000001e-05 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116201+0000", 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1185 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.290350331999999, 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054816747999999998, 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1185 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.34e-07 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061634+0000", 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.996e-06 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061641+0000", 2024-09-22T01:01:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.455e-06 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061645+0000", 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6979999999999997e-06 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116274+0000", 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054628659000000003 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116363+0000", 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8721000000000007e-05 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116449+0000", 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5785000000000006e-05 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1186 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.290330243, 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055030429999999998, 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1186 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061662+0000", 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1700000000000003e-06 2024-09-22T01:01:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061669+0000", 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4109999999999999e-06 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061674+0000", 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999999e-06 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116496+0000", 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054821684000000002 2024-09-22T01:01:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116585+0000", 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9276000000000004e-05 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116682+0000", 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7386000000000006e-05 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1187 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.290300223999999, 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055229227999999998, 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1187 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061685+0000", 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.289e-06 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061691+0000", 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7710000000000001e-06 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061702+0000", 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0443e-05 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116727+0000", 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055024977000000003 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116811+0000", 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4524000000000006e-05 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116911+0000", 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9802000000000004e-05 2024-09-22T01:01:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1282 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.289323639999999, 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054493779999999999, 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1282 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062660+0000", 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6500000000000008e-07 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062664+0000", 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2080000000000002e-06 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062676+0000", 2024-09-22T01:01:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.216e-05 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062684+0000", 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.576e-06 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116948+0000", 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054264616000000002 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117035+0000", 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7331999999999997e-05 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117152+0000", 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011692300000000001 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1283 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.289285778, 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054680980999999997, 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1283 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062697+0000", 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3900000000000005e-07 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062700+0000", 2024-09-22T01:01:14.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062708+0000", 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8019999999999992e-06 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062713+0000", 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609999999999998e-06 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117187+0000", 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054473426999999998 2024-09-22T01:01:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117270+0000", 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3353e-05 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117377+0000", 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010757899999999999 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.386 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1284 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.289260036000002, 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054881757000000003, 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1284 2024-09-22T01:01:14.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062723+0000", 2024-09-22T01:01:14.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062725+0000", 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062732+0000", 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062737+0000", 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2580000000000003e-06 2024-09-22T01:01:14.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117413+0000", 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054675834 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117498+0000", 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4753000000000007e-05 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117604+0000", 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106363 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1285 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.289237966000002, 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055095997000000001, 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1285 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062745+0000", 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062747+0000", 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0760000000000001e-06 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062754+0000", 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.844e-06 2024-09-22T01:01:14.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062759+0000", 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9799999999999998e-06 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117639+0000", 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054880802999999999 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117723+0000", 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3746999999999996e-05 2024-09-22T01:01:14.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117840+0000", 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011718900000000001 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1286 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.289203716999999, 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055282537, 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1286 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062782+0000", 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.835e-06 2024-09-22T01:01:14.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062790+0000", 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.055e-06 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062796+0000", 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117873+0000", 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055077470000000003 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117951+0000", 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118061+0000", 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110027 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1287 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.289178491000001, 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055478658, 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1287 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:01:14.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062806+0000", 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.142e-06 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062813+0000", 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.613e-06 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062818+0000", 2024-09-22T01:01:14.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118096+0000", 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055277739999999999 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118180+0000", 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4795000000000004e-05 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118282+0000", 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101974 2024-09-22T01:01:14.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1288 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.289154497999998, 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055678514999999998, 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1288 2024-09-22T01:01:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1199999999999998e-07 2024-09-22T01:01:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062830+0000", 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.114e-06 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062838+0000", 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8599999999999993e-06 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062843+0000", 2024-09-22T01:01:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118315+0000", 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055471883999999999 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118402+0000", 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6992999999999999e-05 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118506+0000", 2024-09-22T01:01:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104238 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1289 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.289121310999999, 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055873772000000002, 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1289 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063911+0000", 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010504500000000001 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063923+0000", 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1805e-05 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063947+0000", 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3519999999999998e-05 2024-09-22T01:01:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063964+0000", 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6966e-05 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118539+0000", 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054574947999999998 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118631+0000", 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2238999999999996e-05 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118735+0000", 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103844 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1290 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.287978485, 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054954785999999999, 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1290 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064013+0000", 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:01:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064020+0000", 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0310000000000001e-06 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064044+0000", 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4340000000000001e-05 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064081+0000", 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6720000000000001e-05 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118769+0000", 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054687604000000001 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118852+0000", 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118959+0000", 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106969 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1291 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.287878210999999, 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055078496999999997, 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1291 2024-09-22T01:01:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064105+0000", 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.268e-06 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064154+0000", 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9122999999999997e-05 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064174+0000", 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0057000000000001e-05 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064189+0000", 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4083000000000001e-05 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118992+0000", 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054803604999999998 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119079+0000", 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7109999999999998e-05 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119183+0000", 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103251 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1292 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.287771909, 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055189561999999998, 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1292 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065373+0000", 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001162419 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065382+0000", 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6979999999999993e-06 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065403+0000", 2024-09-22T01:01:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0676e-05 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065418+0000", 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.468e-05 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119218+0000", 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053799877000000003 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119301+0000", 2024-09-22T01:01:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2910000000000004e-05 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119400+0000", 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9302000000000006e-05 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1295 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.286412648999999, 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054395285000000002, 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1295 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065571+0000", 2024-09-22T01:01:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:01:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065575+0000", 2024-09-22T01:01:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:01:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065590+0000", 2024-09-22T01:01:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.456e-05 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065616+0000", 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5935000000000002e-05 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119786+0000", 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054170124 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119862+0000", 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5833000000000004e-05 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119965+0000", 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103281 2024-09-22T01:01:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1296 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1327104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.286350341999999, 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054554510000000001, 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1296 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066090+0000", 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045860600000000002 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066104+0000", 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3883000000000001e-05 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066125+0000", 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0392000000000001e-05 2024-09-22T01:01:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066139+0000", 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3925e-05 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120000+0000", 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053860906 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120084+0000", 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4875000000000003e-05 2024-09-22T01:01:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120186+0000", 2024-09-22T01:01:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010192300000000001 2024-09-22T01:01:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1297 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.285821282000001, 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054246075999999997, 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1297 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066728+0000", 2024-09-22T01:01:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056672199999999997 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066733+0000", 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7400000000000001e-06 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066745+0000", 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.166e-05 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066753+0000", 2024-09-22T01:01:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1620000000000008e-06 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120220+0000", 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05346687 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120303+0000", 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3354000000000002e-05 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120407+0000", 2024-09-22T01:01:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103568 2024-09-22T01:01:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:14.442 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:14.442 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:14.443 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:14.443 INFO:teuthology.orchestra.run.smithi002.stdout:1366 write 0xd49117b thru 0xd49702f (0x5eb5 bytes) 2024-09-22T01:01:14.443 INFO:teuthology.orchestra.run.smithi002.stdout:1367 trunc from 0xd497030 to 0xba0ec72 2024-09-22T01:01:14.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:14.444 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:14.445 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:14.445 INFO:teuthology.orchestra.run.smithi002.stdout:1368 write 0x327bc39 thru 0x3282fbb (0x7383 bytes) 2024-09-22T01:01:14.445 INFO:teuthology.orchestra.run.smithi002.stdout:1369 punch from 0x6f8b2e7 to 0x6f9acfe, (0xfa17 bytes) 2024-09-22T01:01:14.445 INFO:teuthology.orchestra.run.smithi002.stdout:1372 writesame 0x30fd8df thru 0x310651e data_size 0x60(0x8c40 bytes) 2024-09-22T01:01:14.445 INFO:teuthology.orchestra.run.smithi002.stdout:1373 trunc from 0xba0ec72 to 0xe9bb3da 2024-09-22T01:01:14.446 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:14.446 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:14.446 INFO:teuthology.orchestra.run.smithi002.stdout:1374 write 0x629b1d8 thru 0x62a2cc8 (0x7af1 bytes) 2024-09-22T01:01:14.447 INFO:teuthology.orchestra.run.smithi002.stdout:1375 read 0x9e645c0 thru 0x9e6be59 (0x789a bytes) 2024-09-22T01:01:14.447 INFO:teuthology.orchestra.run.smithi002.stdout:1376 read 0xdec98b4 thru 0xdecf66e (0x5dbb bytes) 2024-09-22T01:01:14.447 INFO:teuthology.orchestra.run.smithi002.stdout:1377 trunc from 0xe9bb3da to 0x6901166 2024-09-22T01:01:14.447 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:14.448 INFO:teuthology.orchestra.run.smithi002.stdout:1379 punch from 0xdb0990 to 0xdb9421, (0x8a91 bytes) 2024-09-22T01:01:14.448 INFO:teuthology.orchestra.run.smithi002.stdout:1381 trunc from 0x6901166 to 0x5276f8e 2024-09-22T01:01:14.448 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:14.448 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:14.448 INFO:teuthology.orchestra.run.smithi002.stdout:1382 punch from 0x2a95812 to 0x2a97f5c, (0x274a bytes) 2024-09-22T01:01:14.449 INFO:teuthology.orchestra.run.smithi002.stdout:1384 writesame 0x2c5f10c thru 0x2c6c9ca data_size 0x1(0xd8bf bytes) 2024-09-22T01:01:14.451 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:14.449+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:14.461 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:14.461 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:14.464 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:14.465 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:14.507 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:14.528 INFO:teuthology.orchestra.run.smithi002.stdout:1386 writesame 0xaba749d thru 0xabadaa8 data_size 0x2a(0x660c bytes) 2024-09-22T01:01:14.528 INFO:teuthology.orchestra.run.smithi002.stdout:1388 read 0x2e7166e thru 0x2e73bef (0x2582 bytes) 2024-09-22T01:01:14.529 INFO:teuthology.orchestra.run.smithi002.stdout:1389 write 0x9c6cdaa thru 0x9c7185d (0x4ab4 bytes) 2024-09-22T01:01:14.535 INFO:teuthology.orchestra.run.smithi002.stdout:1390 writesame 0x41b09d4 thru 0x41b8f29 data_size 0x3(0x8556 bytes) 2024-09-22T01:01:14.546 INFO:teuthology.orchestra.run.smithi002.stdout:1391 punch from 0x74a5976 to 0x74a72e1, (0x196b bytes) 2024-09-22T01:01:14.546 INFO:teuthology.orchestra.run.smithi002.stdout:1394 write 0x38c3bd3 thru 0x38d0824 (0xcc52 bytes) 2024-09-22T01:01:14.550 INFO:teuthology.orchestra.run.smithi002.stdout:1395 writesame 0x551fa6c thru 0x552e896 data_size 0x1(0xee2b bytes) 2024-09-22T01:01:14.550 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:14.548+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:14.587 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:14.587 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:14.602 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:14.612 INFO:teuthology.orchestra.run.smithi002.stdout:1396 writesame 0x554bed5 thru 0x5556f4e data_size 0x31(0xb07a bytes) 2024-09-22T01:01:14.615 INFO:teuthology.orchestra.run.smithi002.stdout:1397 punch from 0x611cd21 to 0x612487b, (0x7b5a bytes) 2024-09-22T01:01:14.615 INFO:teuthology.orchestra.run.smithi002.stdout:1398 read 0x7103702 thru 0x7104e88 (0x1787 bytes) 2024-09-22T01:01:14.616 INFO:teuthology.orchestra.run.smithi002.stdout:1400 punch from 0x64e2369 to 0x64eb50e, (0x91a5 bytes) 2024-09-22T01:01:14.616 INFO:teuthology.orchestra.run.smithi002.stdout:1401 write 0x295de4 thru 0x2a45d4 (0xe7f1 bytes) 2024-09-22T01:01:14.618 INFO:teuthology.orchestra.run.smithi002.stdout:1402 read 0x6d3afe8 thru 0x6d42275 (0x728e bytes) 2024-09-22T01:01:14.619 INFO:teuthology.orchestra.run.smithi002.stdout:1403 read 0x15a4c0c thru 0x15a5ee7 (0x12dc bytes) 2024-09-22T01:01:14.641 INFO:teuthology.orchestra.run.smithi002.stdout:1404 write 0xd5ae725 thru 0xd5b5baa (0x7486 bytes) 2024-09-22T01:01:14.648 INFO:teuthology.orchestra.run.smithi002.stdout:1405 read 0x57dbfd9 thru 0x57eb241 (0xf269 bytes) 2024-09-22T01:01:14.649 INFO:teuthology.orchestra.run.smithi002.stdout:1407 writesame 0xbbf0832 thru 0xbbf98c4 data_size 0xd(0x9093 bytes) 2024-09-22T01:01:14.651 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:14.648+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:14.659 INFO:teuthology.orchestra.run.smithi002.stdout:1409 trunc from 0xd5b5bab to 0xca3126a 2024-09-22T01:01:14.675 INFO:teuthology.orchestra.run.smithi002.stdout:1410 write 0x1bb6bb3 thru 0x1bbc7f5 (0x5c43 bytes) 2024-09-22T01:01:14.708 INFO:teuthology.orchestra.run.smithi002.stdout:1411 trunc from 0xca3126a to 0x1199adf 2024-09-22T01:01:14.731 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:14.731 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:14.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:14.781 INFO:teuthology.orchestra.run.smithi002.stdout:1412 write 0xbacc6f7 thru 0xbadc50f (0xfe19 bytes) 2024-09-22T01:01:14.788 INFO:teuthology.orchestra.run.smithi002.stdout:1413 punch from 0x42dc6db to 0x42e14a2, (0x4dc7 bytes) 2024-09-22T01:01:14.792 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:14.793+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:14.801 INFO:teuthology.orchestra.run.smithi002.stdout:1415 trunc from 0xbadc510 to 0x72cc751 2024-09-22T01:01:14.840 INFO:teuthology.orchestra.run.smithi002.stdout:1417 trunc from 0x72cc751 to 0xe5fce5 2024-09-22T01:01:14.853 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:14.852+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:14.858 INFO:teuthology.orchestra.run.smithi002.stdout:1419 read 0x5c67b6 thru 0x5cd1a8 (0x69f3 bytes) 2024-09-22T01:01:14.859 INFO:teuthology.orchestra.run.smithi002.stdout:1420 read 0x14310 thru 0x1cb7a (0x886b bytes) 2024-09-22T01:01:14.861 INFO:teuthology.orchestra.run.smithi002.stdout:1421 punch from 0xc225c to 0xcd746, (0xb4ea bytes) 2024-09-22T01:01:14.867 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:14.867 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:14.882 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:14.898 INFO:teuthology.orchestra.run.smithi002.stdout:1422 write 0x998cd08 thru 0x999c6ff (0xf9f8 bytes) 2024-09-22T01:01:14.905 INFO:teuthology.orchestra.run.smithi002.stdout:1423 writesame 0x592fb8c thru 0x5936c30 data_size 0x1(0x70a5 bytes) 2024-09-22T01:01:14.934 INFO:teuthology.orchestra.run.smithi002.stdout:1424 read 0x1d086e8 thru 0x1d10999 (0x82b2 bytes) 2024-09-22T01:01:14.959 INFO:teuthology.orchestra.run.smithi002.stdout:1425 write 0xd8a8cb3 thru 0xd8b6e68 (0xe1b6 bytes) 2024-09-22T01:01:14.965 INFO:teuthology.orchestra.run.smithi002.stdout:1426 writesame 0x1d095cf thru 0x1d0b959 data_size 0x9(0x238b bytes) 2024-09-22T01:01:14.967 INFO:teuthology.orchestra.run.smithi002.stdout:1428 punch from 0x63ffd4d to 0x6405d8a, (0x603d bytes) 2024-09-22T01:01:14.967 INFO:teuthology.orchestra.run.smithi002.stdout:1429 read 0x3bf086e thru 0x3bf4fac (0x473f bytes) 2024-09-22T01:01:14.968 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:14.969+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:14.988 INFO:teuthology.orchestra.run.smithi002.stdout:1431 trunc from 0xd8b6e69 to 0x61a5691 2024-09-22T01:01:15.015 INFO:teuthology.orchestra.run.smithi002.stdout:1433 clone 13 order 20 su 32768 sc 15 2024-09-22T01:01:15.043 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:15.043 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:15.055 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:15.052+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:15.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:15.156 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:15.157+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:15.222 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:15.222 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:15.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:15.256 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:15.252+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:15.357 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:15.356+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:15.407 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:15.407 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:15.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:15.468 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:15.469+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:15.544 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:15.544 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:15.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:15.558 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:15.556+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:15.659 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:15.660+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:15.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:15.760 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:15.756+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:15.860 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:15.861+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:15.874 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:15.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:01:15.961 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:15.961+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:16.014 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone12 from 0x61a5691 (overlap 0xe5fce5) to 0x0 2024-09-22T01:01:16.017 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:16.017 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:16.033 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:16.062 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:16.061+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:16.144 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-22T01:01:16.162 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:16.160+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:16.177 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:16.177 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:16.177 INFO:teuthology.orchestra.run.smithi002.stdout:1434 trunc from 0x61a5691 to 0x86732f6 2024-09-22T01:01:16.193 INFO:teuthology.orchestra.run.smithi002.stdout:1435 read 0x616af6e thru 0x6175a11 (0xaaa4 bytes) 2024-09-22T01:01:16.195 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:16.206 INFO:teuthology.orchestra.run.smithi002.stdout:1437 writesame 0xabb48bc thru 0xabb61a2 data_size 0x19(0x18e7 bytes) 2024-09-22T01:01:16.220 INFO:teuthology.orchestra.run.smithi002.stdout:1440 write 0xde763a7 thru 0xde7bad2 (0x572c bytes) 2024-09-22T01:01:16.229 INFO:teuthology.orchestra.run.smithi002.stdout:1441 write 0xbf995a2 thru 0xbf9d92c (0x438b bytes) 2024-09-22T01:01:16.236 INFO:teuthology.orchestra.run.smithi002.stdout:1443 punch from 0x8350a94 to 0x8351599, (0xb05 bytes) 2024-09-22T01:01:16.236 INFO:teuthology.orchestra.run.smithi002.stdout:1444 write 0x4ddf614 thru 0x4ddfde5 (0x7d2 bytes) 2024-09-22T01:01:16.250 INFO:teuthology.orchestra.run.smithi002.stdout:1445 read 0x9e4e552 thru 0x9e51197 (0x2c46 bytes) 2024-09-22T01:01:16.250 INFO:teuthology.orchestra.run.smithi002.stdout:1446 read 0x15d96c4 thru 0x15e212c (0x8a69 bytes) 2024-09-22T01:01:16.264 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:16.265+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:16.272 INFO:teuthology.orchestra.run.smithi002.stdout:1449 trunc from 0xde7bad3 to 0x1036152 2024-09-22T01:01:16.341 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:16.341 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:16.356 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:16.397 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:16.396+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:16.462 INFO:teuthology.orchestra.run.smithi002.stdout:1450 write 0x6a173e thru 0x6b037a (0xec3d bytes) 2024-09-22T01:01:16.465 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:16.464+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:16.465 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:16.496 INFO:teuthology.orchestra.run.smithi002.stdout:1451 write 0x168c075 thru 0x168e182 (0x210e bytes) 2024-09-22T01:01:16.516 INFO:teuthology.orchestra.run.smithi002.stdout:1452 write 0x7ad8744 thru 0x7ade97a (0x6237 bytes) 2024-09-22T01:01:16.523 INFO:teuthology.orchestra.run.smithi002.stdout:1454 read 0xbf98a5 thru 0xc022ab (0x8a07 bytes) 2024-09-22T01:01:16.524 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:16.524 INFO:teuthology.orchestra.run.smithi002.stdout:1456 read 0x42ae2c7 thru 0x42b0b99 (0x28d3 bytes) 2024-09-22T01:01:16.525 INFO:teuthology.orchestra.run.smithi002.stdout:1457 writesame 0x5fd4132 thru 0x5fe2a0a data_size 0x1(0xe8d9 bytes) 2024-09-22T01:01:16.536 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:16.536 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:16.552 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:16.577 INFO:teuthology.orchestra.run.smithi002.stdout:1458 write 0x1915a27 thru 0x19202dd (0xa8b7 bytes) 2024-09-22T01:01:16.577 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:16.576+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:16.580 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:16.580 INFO:teuthology.orchestra.run.smithi002.stdout:1460 read 0x94bb4b thru 0x951f5e (0x6414 bytes) 2024-09-22T01:01:16.581 INFO:teuthology.orchestra.run.smithi002.stdout:1462 write 0x3b479c2 thru 0x3b4b0dd (0x371c bytes) 2024-09-22T01:01:16.604 INFO:teuthology.orchestra.run.smithi002.stdout:1466 write 0xca2b6c1 thru 0xca31beb (0x652b bytes) 2024-09-22T01:01:16.609 INFO:teuthology.orchestra.run.smithi002.stdout:1468 read 0xc5f913d thru 0xc5fd196 (0x405a bytes) 2024-09-22T01:01:16.609 INFO:teuthology.orchestra.run.smithi002.stdout:1469 punch from 0x21d4e65 to 0x21e1420, (0xc5bb bytes) 2024-09-22T01:01:16.609 INFO:teuthology.orchestra.run.smithi002.stdout:1470 write 0xa110820 thru 0xa114bae (0x438f bytes) 2024-09-22T01:01:16.614 INFO:teuthology.orchestra.run.smithi002.stdout:1471 read 0x13d9194 thru 0x13e8624 (0xf491 bytes) 2024-09-22T01:01:16.614 INFO:teuthology.orchestra.run.smithi002.stdout:1472 writesame 0x8aa15e9 thru 0x8aa1f7b data_size 0x13(0x993 bytes) 2024-09-22T01:01:16.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:16.618 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:16.620 INFO:teuthology.orchestra.run.smithi002.stdout:1473 write 0x3f21534 thru 0x3f2a2ef (0x8dbc bytes) 2024-09-22T01:01:16.627 INFO:teuthology.orchestra.run.smithi002.stdout:1474 write 0x2000b0c thru 0x2004957 (0x3e4c bytes) 2024-09-22T01:01:16.631 INFO:teuthology.orchestra.run.smithi002.stdout:1475 writesame 0x44b8fe7 thru 0x44c5bd7 data_size 0x3(0xcbf1 bytes) 2024-09-22T01:01:16.637 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:16.637 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:16.640 INFO:teuthology.orchestra.run.smithi002.stdout:1476 write 0x38511b5 thru 0x3860800 (0xf64c bytes) 2024-09-22T01:01:16.650 INFO:teuthology.orchestra.run.smithi002.stdout:1478 write 0x79c3e4f thru 0x79cc94d (0x8aff bytes) 2024-09-22T01:01:16.656 INFO:teuthology.orchestra.run.smithi002.stdout:1479 write 0xb9e6bc8 thru 0xb9f39f2 (0xce2b bytes) 2024-09-22T01:01:16.666 INFO:teuthology.orchestra.run.smithi002.stdout:1480 writesame 0xb5c1bed thru 0xb5ce64b data_size 0x1(0xca5f bytes) 2024-09-22T01:01:16.666 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:16.664+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:16.670 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:16.670 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:16.696 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:16.696 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:16.716 INFO:teuthology.orchestra.run.smithi002.stdout:1481 punch from 0x3dfee48 to 0x3e0b883, (0xca3b bytes) 2024-09-22T01:01:16.716 INFO:teuthology.orchestra.run.smithi002.stdout:1482 writesame 0xbfe0336 thru 0xbfecc04 data_size 0x1(0xc8cf bytes) 2024-09-22T01:01:16.717 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:16.717 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:16.718 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:16.722 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:16.722 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:16.722 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:16.722 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:16.722 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:16.737 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:16.739 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:16.740 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:16.761 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:16.761 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:16.765 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:16.765 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:16.768 INFO:teuthology.orchestra.run.smithi002.stdout:1483 writesame 0xcb52e7c thru 0xcb58e73 data_size 0x53(0x5ff8 bytes) 2024-09-22T01:01:16.768 INFO:teuthology.orchestra.run.smithi002.stdout:1486 writesame 0x909333b thru 0x909fc03 data_size 0x7(0xc8c9 bytes) 2024-09-22T01:01:16.769 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:16.764+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:16.776 INFO:teuthology.orchestra.run.smithi002.stdout:1488 writesame 0x929ffe8 thru 0x92aa046 data_size 0x55(0xa05f bytes) 2024-09-22T01:01:16.784 INFO:teuthology.orchestra.run.smithi002.stdout:1490 write 0xfbb9f2 thru 0xfc193c (0x5f4b bytes) 2024-09-22T01:01:16.786 INFO:teuthology.orchestra.run.smithi002.stdout:1491 punch from 0x8a7d701 to 0x8a7e11a, (0xa19 bytes) 2024-09-22T01:01:16.798 INFO:teuthology.orchestra.run.smithi002.stdout:1492 trunc from 0xcb58e74 to 0x5dd8a3a 2024-09-22T01:01:16.822 INFO:teuthology.orchestra.run.smithi002.stdout:1494 punch from 0x814e4e to 0x815e8a, (0x103c bytes) 2024-09-22T01:01:16.822 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:16.822 INFO:teuthology.orchestra.run.smithi002.stdout:1496 read 0x543c533 thru 0x54448d5 (0x83a3 bytes) 2024-09-22T01:01:16.822 INFO:teuthology.orchestra.run.smithi002.stdout:1497 write 0x313f282 thru 0x314c012 (0xcd91 bytes) 2024-09-22T01:01:16.839 INFO:teuthology.orchestra.run.smithi002.stdout:1498 write 0x8675947 thru 0x867d95b (0x8015 bytes) 2024-09-22T01:01:16.846 INFO:teuthology.orchestra.run.smithi002.stdout:1499 read 0x6bb0d5a thru 0x6bbb752 (0xa9f9 bytes) 2024-09-22T01:01:16.846 INFO:teuthology.orchestra.run.smithi002.stdout:1500 write 0x807d8a2 thru 0x807ef10 (0x166f bytes) 2024-09-22T01:01:16.863 INFO:teuthology.orchestra.run.smithi002.stdout:1501 writesame 0xae59104 thru 0xae5eb95 data_size 0x1(0x5a92 bytes) 2024-09-22T01:01:16.864 INFO:teuthology.orchestra.run.smithi002.stdout:1502 write 0x523e7d2 thru 0x5244c37 (0x6466 bytes) 2024-09-22T01:01:16.869 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:16.869+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:16.871 INFO:teuthology.orchestra.run.smithi002.stdout:1503 write 0xac96da2 thru 0xaca036c (0x95cb bytes) 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796775180000001, 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:16.881 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796756321, 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:16.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796738135999998, 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.886 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796720348000001, 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:16.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:16.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:16.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.79670179, 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796683737999999, 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:16.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:16.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:16.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796666388999999, 2024-09-22T01:01:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:16.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:16.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796644436000001, 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.907 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:16.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796626901, 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.915 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796603115, 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:16.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:16.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796585253, 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:16.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:16.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796558015999999, 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:16.924 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:16.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.928 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796539311, 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:16.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796515813999999, 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:16.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:16.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.796491844999998, 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:16.937 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:16.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.79647263, 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.795468079999999, 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.946 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.793014891999999, 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:16.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.792992812000001, 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:16.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:16.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.792962126999999, 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:16.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:16.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:16.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:16.963 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:16.964 INFO:teuthology.orchestra.run.smithi002.stdout:1504 read 0x558a2e8 thru 0x5598c75 (0xe98e bytes) 2024-09-22T01:01:16.964 INFO:teuthology.orchestra.run.smithi002.stdout:1505 read 0x458574b thru 0x458df1c (0x87d2 bytes) 2024-09-22T01:01:16.964 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:01:16.965 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:16.966 INFO:teuthology.orchestra.run.smithi002.stdout:1506 writesame 0xdd1b4e6 thru 0xdd1ed08 data_size 0x1(0x3823 bytes) 2024-09-22T01:01:16.966 INFO:teuthology.orchestra.run.smithi002.stdout:1507 write 0xba9c7c5 thru 0xba9e773 (0x1faf bytes) 2024-09-22T01:01:16.966 INFO:teuthology.orchestra.run.smithi002.stdout:1508 writesame 0x8ffb998 thru 0x9001451 data_size 0x62(0x5aba bytes) 2024-09-22T01:01:16.966 INFO:teuthology.orchestra.run.smithi002.stdout:1509 trunc from 0xdd1ed09 to 0x1c5bd8 2024-09-22T01:01:16.970 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:16.969+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:17.071 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:17.068+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:17.098 INFO:teuthology.orchestra.run.smithi002.stdout:1510 write 0x215966a thru 0x2167d48 (0xe6df bytes) 2024-09-22T01:01:17.106 INFO:teuthology.orchestra.run.smithi002.stdout:1511 read 0x12451df thru 0x124c4c4 (0x72e6 bytes) 2024-09-22T01:01:17.127 INFO:teuthology.orchestra.run.smithi002.stdout:1513 write 0x882e80c thru 0x883a344 (0xbb39 bytes) 2024-09-22T01:01:17.150 INFO:teuthology.orchestra.run.smithi002.stdout:1514 write 0x9df742e thru 0x9dfe631 (0x7204 bytes) 2024-09-22T01:01:17.153 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:17.153 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:17.153 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:17.153 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:17.159 INFO:teuthology.orchestra.run.smithi002.stdout:1515 read 0x99fed4d thru 0x9a0dbd3 (0xee87 bytes) 2024-09-22T01:01:17.159 INFO:teuthology.orchestra.run.smithi002.stdout:1516 write 0xaedb9c thru 0xaf3df3 (0x6258 bytes) 2024-09-22T01:01:17.162 INFO:teuthology.orchestra.run.smithi002.stdout:1517 write 0x8d8ee50 thru 0x8d946e3 (0x5894 bytes) 2024-09-22T01:01:17.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:17.170 INFO:teuthology.orchestra.run.smithi002.stdout:1518 punch from 0xae38cf to 0xae861f, (0x4d50 bytes) 2024-09-22T01:01:17.171 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:17.169+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:17.176 INFO:teuthology.orchestra.run.smithi002.stdout:1519 trunc from 0x9dfe632 to 0x5fa21e4 2024-09-22T01:01:17.194 INFO:teuthology.orchestra.run.smithi002.stdout:1520 read 0x5023b6b thru 0x502fd61 (0xc1f7 bytes) 2024-09-22T01:01:17.197 INFO:teuthology.orchestra.run.smithi002.stdout:1521 trunc from 0x5fa21e4 to 0x7c312f9 2024-09-22T01:01:17.216 INFO:teuthology.orchestra.run.smithi002.stdout:1522 trunc from 0x7c312f9 to 0x2ef5e0e 2024-09-22T01:01:17.254 INFO:teuthology.orchestra.run.smithi002.stdout:1524 write 0xc3987fe thru 0xc399b6f (0x1372 bytes) 2024-09-22T01:01:17.260 INFO:teuthology.orchestra.run.smithi002.stdout:1525 write 0x7d54abc thru 0x7d57d0a (0x324f bytes) 2024-09-22T01:01:17.271 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:17.268+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:17.281 INFO:teuthology.orchestra.run.smithi002.stdout:1526 write 0xd18bcc7 thru 0xd19a85e (0xeb98 bytes) 2024-09-22T01:01:17.293 INFO:teuthology.orchestra.run.smithi002.stdout:1527 punch from 0xc2d178f to 0xc2d99d7, (0x8248 bytes) 2024-09-22T01:01:17.293 INFO:teuthology.orchestra.run.smithi002.stdout:1528 read 0xcc1d994 thru 0xcc1f612 (0x1c7f bytes) 2024-09-22T01:01:17.293 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.308 INFO:teuthology.orchestra.run.smithi002.stdout:1531 trunc from 0xd19a85f to 0x3bce9f7 2024-09-22T01:01:17.328 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:17.329 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:17.329 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:17.329 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:17.329 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:17.331 INFO:teuthology.orchestra.run.smithi002.stdout:1532 punch from 0x10fccf4 to 0x110b419, (0xe725 bytes) 2024-09-22T01:01:17.331 INFO:teuthology.orchestra.run.smithi002.stdout:1533 writesame 0x2dbe85b thru 0x2dcc6d3 data_size 0xd(0xde79 bytes) 2024-09-22T01:01:17.342 INFO:teuthology.orchestra.run.smithi002.stdout:1534 write 0x23dadfd thru 0x23dd6bc (0x28c0 bytes) 2024-09-22T01:01:17.345 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:17.348 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.348 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.360 INFO:teuthology.orchestra.run.smithi002.stdout:1537 write 0x4f2f4b8 thru 0x4f343bc (0x4f05 bytes) 2024-09-22T01:01:17.373 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:17.373+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:17.392 INFO:teuthology.orchestra.run.smithi002.stdout:1538 write 0xcdd8e61 thru 0xcde33e4 (0xa584 bytes) 2024-09-22T01:01:17.399 INFO:teuthology.orchestra.run.smithi002.stdout:1541 read 0xb2d7376 thru 0xb2e18dc (0xa567 bytes) 2024-09-22T01:01:17.399 INFO:teuthology.orchestra.run.smithi002.stdout:1542 read 0xbdeeea9 thru 0xbdf057b (0x16d3 bytes) 2024-09-22T01:01:17.399 INFO:teuthology.orchestra.run.smithi002.stdout:1543 write 0x86dca85 thru 0x86e2d85 (0x6301 bytes) 2024-09-22T01:01:17.407 INFO:teuthology.orchestra.run.smithi002.stdout:1545 punch from 0x3664a82 to 0x36670a4, (0x2622 bytes) 2024-09-22T01:01:17.407 INFO:teuthology.orchestra.run.smithi002.stdout:1546 read 0x31ab4d2 thru 0x31b9404 (0xdf33 bytes) 2024-09-22T01:01:17.407 INFO:teuthology.orchestra.run.smithi002.stdout:1548 punch from 0x7a8bea1 to 0x7a97707, (0xb866 bytes) 2024-09-22T01:01:17.407 INFO:teuthology.orchestra.run.smithi002.stdout:1549 read 0x231f1b0 thru 0x2328d37 (0x9b88 bytes) 2024-09-22T01:01:17.407 INFO:teuthology.orchestra.run.smithi002.stdout:1550 read 0x64e871b thru 0x64f07d4 (0x80ba bytes) 2024-09-22T01:01:17.407 INFO:teuthology.orchestra.run.smithi002.stdout:1551 write 0x43a9e80 thru 0x43b58d0 (0xba51 bytes) 2024-09-22T01:01:17.434 INFO:teuthology.orchestra.run.smithi002.stdout:1554 trunc from 0xcde33e5 to 0x1c7f91 2024-09-22T01:01:17.473 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:17.473+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:17.487 INFO:teuthology.orchestra.run.smithi002.stdout:1555 writesame 0x5d70e99 thru 0x5d76893 data_size 0x11(0x59fb bytes) 2024-09-22T01:01:17.487 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.488 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.488 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.488 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.488 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.492 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:17.492 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4235.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.562372066999998, 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015255189999999999, 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930031+0000", 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930032+0000", 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-07 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930049+0000", 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7669999999999999e-05 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930072+0000", 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2232e-05 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930090+0000", 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8491999999999998e-05 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930185+0000", 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4500999999999995e-05 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.930236+0000", 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1798000000000001e-05 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.931471+0000", 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001234228 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:37.931557+0000", 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5853999999999995e-05 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:1943 3.0 e28/22 3:046f320d:::rbd_data.117c1b47bdd2.000000000000003b:head v 28'6, mlcod=25'3)", 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.362286080000001, 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017318469999999999, 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130117+0000", 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130118+0000", 2024-09-22T01:01:17.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3900000000000004e-07 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130185+0000", 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7076999999999995e-05 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130201+0000", 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6320999999999998e-05 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130210+0000", 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9279999999999999e-06 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130349+0000", 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013925799999999999 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.130372+0000", 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2660999999999999e-05 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.131801+0000", 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001429034 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.131849+0000", 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7729000000000003e-05 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:7990 3.4 e32/22 3:2775b4d4:::rbd_data.117ce1c46ae.0000000000000032:head v 32'16, mlcod=31'12)", 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:58.941629+0000", 2024-09-22T01:01:17.501 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.550773602, 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014661279999999999, 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941629+0000", 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941629+0000", 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941631+0000", 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3850000000000001e-06 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941641+0000", 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9559999999999992e-06 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941667+0000", 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5965999999999998e-05 2024-09-22T01:01:17.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941677+0000", 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0234999999999999e-05 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941743+0000", 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6192000000000006e-05 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.941766+0000", 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2429000000000002e-05 2024-09-22T01:01:17.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.943039+0000", 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012731070000000001 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.943096+0000", 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6857999999999997e-05 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8071 3.4 e32/22 3:230ffd57:::rbd_data.117ce1c46ae.0000000000000018:head v 32'17, mlcod=32'14)", 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:58.959772+0000", 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.532630727000001, 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017058590000000001, 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959772+0000", 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959772+0000", 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959773+0000", 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5000000000000001e-07 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959955+0000", 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018200399999999999 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959969+0000", 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3857e-05 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.959978+0000", 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9409999999999996e-06 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.960031+0000", 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3402000000000001e-05 2024-09-22T01:01:17.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.960047+0000", 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5231e-05 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.961427+0000", 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001380559 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:58.961478+0000", 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1014999999999999e-05 2024-09-22T01:01:17.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8964 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'20, mlcod=32'17)", 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.355879753, 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077894110000000004, 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136525+0000", 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3540000000000001e-06 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137650+0000", 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011257190000000001 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137671+0000", 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0560999999999998e-05 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137680+0000", 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-22T01:01:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137763+0000", 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2472999999999995e-05 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137793+0000", 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-05 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144267+0000", 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064739139999999999 2024-09-22T01:01:17.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144313+0000", 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5624999999999998e-05 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8968 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'21, mlcod=32'17)", 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.351453708000001, 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004894771, 2024-09-22T01:01:17.514 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140952+0000", 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1849999999999998e-06 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.141985+0000", 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001032984 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142001+0000", 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6776999999999999e-05 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142010+0000", 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4640000000000006e-06 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142101+0000", 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1312999999999998e-05 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142119+0000", 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8202e-05 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145810+0000", 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036909970000000001 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145844+0000", 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3849000000000001e-05 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.518 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8991 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'22, mlcod=32'20)", 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.231258+0000", 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.261145340999999, 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017482730000000001, 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231258+0000", 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231258+0000", 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231260+0000", 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4439999999999998e-06 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231305+0000", 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5118999999999999e-05 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231341+0000", 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5500000000000002e-05 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231362+0000", 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0696e-05 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231527+0000", 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165043 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.231587+0000", 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0862000000000001e-05 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.232906+0000", 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00131909 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.233006+0000", 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9518999999999994e-05 2024-09-22T01:01:17.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(3.4 epoch 32/22 rep_tid 55 entries 32'26 (0'0) error 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head by client.4476.0:9032 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 32'24)", 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.304780+0000", 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.187623359, 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002035411, 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304780+0000", 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304780+0000", 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304783+0000", 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304809+0000", 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6207000000000002e-05 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304875+0000", 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6415999999999996e-05 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.304917+0000", 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1539999999999999e-05 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305262+0000", 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00034516000000000001 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.306815+0000", 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015529680000000001 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(3.4 epoch 32/22 rep_tid 56 entries 32'27 (0'0) error 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head by client.4476.0:9033 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 32'24)", 2024-09-22T01:01:17.526 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.305934+0000", 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.186469144, 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016401110000000001, 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305934+0000", 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305934+0000", 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305936+0000", 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8059999999999999e-06 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305956+0000", 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9993999999999999e-05 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.305999+0000", 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3495999999999998e-05 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.306035+0000", 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5565999999999998e-05 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.306266+0000", 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000231435 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.307574+0000", 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001307814 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9048 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'28, mlcod=32'26)", 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.320562+0000", 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.171841405999999, 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019977699999999998, 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.530 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320562+0000", 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320562+0000", 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320564+0000", 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.934e-06 2024-09-22T01:01:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320627+0000", 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3546000000000004e-05 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320654+0000", 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7117000000000001e-05 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320670+0000", 2024-09-22T01:01:17.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5311999999999998e-05 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320763+0000", 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3202000000000007e-05 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.320798+0000", 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5398999999999998e-05 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.322477+0000", 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016784739999999999 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.322559+0000", 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2786000000000003e-05 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9061 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'30, mlcod=32'28)", 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.337997+0000", 2024-09-22T01:01:17.534 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.154405681, 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001560664, 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.337997+0000", 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.337997+0000", 2024-09-22T01:01:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.337999+0000", 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.736e-06 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338078+0000", 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8652999999999994e-05 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338111+0000", 2024-09-22T01:01:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2863e-05 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338128+0000", 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7017e-05 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338225+0000", 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7083000000000005e-05 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.338259+0000", 2024-09-22T01:01:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4400000000000003e-05 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.339432+0000", 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00117303 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.339558+0000", 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012588200000000001 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9077 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'31, mlcod=32'29)", 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.097129333000002, 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034796760000000001, 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395277+0000", 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8600000000000001e-06 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395294+0000", 2024-09-22T01:01:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7513e-05 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395337+0000", 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2749e-05 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395358+0000", 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1097000000000001e-05 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395548+0000", 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189829 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395593+0000", 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4919000000000001e-05 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398634+0000", 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003041373 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398753+0000", 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119336 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9083 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'32, mlcod=32'30)", 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.091348052000001, 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002882763, 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401057+0000", 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.677e-06 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401069+0000", 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2673e-05 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401107+0000", 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7200000000000003e-05 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401122+0000", 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5203e-05 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401251+0000", 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012935900000000001 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401287+0000", 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5633000000000003e-05 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.403853+0000", 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025658389999999999 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.403938+0000", 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5179000000000006e-05 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9847 3.0 e33/22 3:063de76a:::rbd_data.117ccb654ee9.0000000000000012:head v 33'23, mlcod=31'20)", 2024-09-22T01:01:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.301313+0000", 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.191089593000001, 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001906673, 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301313+0000", 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301313+0000", 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301314+0000", 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6299999999999993e-07 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301366+0000", 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1748e-05 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301390+0000", 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3796999999999999e-05 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301403+0000", 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3200000000000001e-05 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301531+0000", 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000128045 2024-09-22T01:01:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.301570+0000", 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8856999999999997e-05 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.303099+0000", 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001528611 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.303220+0000", 2024-09-22T01:01:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000121452 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9855 3.0 e33/22 3:063de76a:::rbd_data.117ccb654ee9.0000000000000012:head v 33'25, mlcod=33'23)", 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.337905+0000", 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.154498160999999, 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018437970000000001, 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337905+0000", 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337905+0000", 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337907+0000", 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0999999999999998e-06 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337962+0000", 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5186999999999999e-05 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.337995+0000", 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3015999999999997e-05 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.338020+0000", 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5103e-05 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.338186+0000", 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016595600000000001 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.338255+0000", 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8425999999999998e-05 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.339667+0000", 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014121959999999999 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.339749+0000", 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1812999999999997e-05 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9864 3.0 e33/22 3:063de76a:::rbd_data.117ccb654ee9.0000000000000012:head v 33'27, mlcod=33'25)", 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.365897+0000", 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.126506353, 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001575611, 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365897+0000", 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365897+0000", 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365899+0000", 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8560000000000001e-06 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365914+0000", 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5630000000000001e-05 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365956+0000", 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1486999999999998e-05 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.365975+0000", 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9383000000000001e-05 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.366175+0000", 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020039799999999999 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.366230+0000", 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4429999999999999e-05 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.367385+0000", 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011547739999999999 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.367472+0000", 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7652999999999996e-05 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9941 3.0 e33/22 3:063de76a:::rbd_data.117ccb654ee9.0000000000000012:head v 33'29, mlcod=33'27)", 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.484122+0000", 2024-09-22T01:01:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.008280669000001, 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018964240000000001, 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484122+0000", 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484122+0000", 2024-09-22T01:01:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484125+0000", 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5660000000000001e-06 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484144+0000", 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9029999999999999e-05 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484186+0000", 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1489999999999997e-05 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484206+0000", 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0956999999999999e-05 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484424+0000", 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021716800000000001 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.484478+0000", 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4725999999999997e-05 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.485960+0000", 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014821439999999999 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.486019+0000", 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8343e-05 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9958 3.9 e33/22 3:91af50a5:::rbd_data.117ccb654ee9.0000000000000002:head v 33'15, mlcod=31'13)", 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:00.497892+0000", 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.994510978000001, 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015364160000000001, 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497892+0000", 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497892+0000", 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497894+0000", 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6530000000000001e-06 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497918+0000", 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3862999999999998e-05 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497947+0000", 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9442999999999999e-05 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.497961+0000", 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.376e-05 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.498039+0000", 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8659999999999996e-05 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.498063+0000", 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3591000000000002e-05 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.499380+0000", 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001316953 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:00.499429+0000", 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8492999999999998e-05 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:14402 3.4 e39/22 3:25583b41:::rbd_data.117c1bff5a72.000000000000000d:head v 39'61, mlcod=39'59)", 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:08.933151+0000", 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.5592520450000009, 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015489220000000001, 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933151+0000", 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933151+0000", 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933153+0000", 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.81e-06 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933188+0000", 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4903000000000001e-05 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933209+0000", 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1214999999999999e-05 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933218+0000", 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4460000000000007e-06 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933290+0000", 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1440999999999996e-05 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.933317+0000", 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7053999999999999e-05 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.934609+0000", 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012921059999999999 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.934700+0000", 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0946999999999995e-05 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:14416 3.4 e39/22 3:25583b41:::rbd_data.117c1bff5a72.000000000000000d:head v 39'62, mlcod=39'60)", 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:08.950669+0000", 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.5417343680000002, 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001939493, 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950669+0000", 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950669+0000", 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950670+0000", 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7129999999999999e-06 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950685+0000", 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4460999999999999e-05 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950716+0000", 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1028999999999997e-05 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950729+0000", 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2787e-05 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950869+0000", 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000139989 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.950935+0000", 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6767999999999995e-05 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.952563+0000", 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016272190000000001 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:08.952608+0000", 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5527e-05 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:17.577 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:17.578 INFO:teuthology.orchestra.run.smithi002.stdout:1562 trunc from 0x5d76894 to 0xd9d211a 2024-09-22T01:01:17.579 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1563 punch from 0x8378802 to 0x837b921, (0x311f bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1564 read 0x3079a3c thru 0x308609b (0xc660 bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1566 punch from 0x93049da to 0x93143cd, (0xf9f3 bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1568 punch from 0x58e4627 to 0x58ed270, (0x8c49 bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1569 write 0xcaff817 thru 0xcb0997a (0xa164 bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1572 read 0x45c21eb thru 0x45c4827 (0x263d bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1573 punch from 0x53e47c7 to 0x53e7d8a, (0x35c3 bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1574 read 0x21d6666 thru 0x21e3561 (0xcefc bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1576 read 0x9a3ee0e thru 0x9a4dadb (0xecce bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1578 read 0x3c5ee41 thru 0x3c67c8b (0x8e4b bytes) 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1579 trunc from 0xd9d211a to 0x44fdf29 2024-09-22T01:01:17.581 INFO:teuthology.orchestra.run.smithi002.stdout:1580 trunc from 0x44fdf29 to 0xda78d00 2024-09-22T01:01:17.582 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:17.577+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:17.593 INFO:teuthology.orchestra.run.smithi002.stdout:1581 write 0x5ae065f thru 0x5ae82de (0x7c80 bytes) 2024-09-22T01:01:17.601 INFO:teuthology.orchestra.run.smithi002.stdout:1582 read 0xae6bd53 thru 0xae6fab2 (0x3d60 bytes) 2024-09-22T01:01:17.601 INFO:teuthology.orchestra.run.smithi002.stdout:1583 writesame 0x3102aa1 thru 0x311271e data_size 0x51(0xfc7e bytes) 2024-09-22T01:01:17.610 INFO:teuthology.orchestra.run.smithi002.stdout:1584 write 0x5b2f28d thru 0x5b357de (0x6552 bytes) 2024-09-22T01:01:17.620 INFO:teuthology.orchestra.run.smithi002.stdout:1585 writesame 0x44ac782 thru 0x44b1568 data_size 0x31(0x4de7 bytes) 2024-09-22T01:01:17.627 INFO:teuthology.orchestra.run.smithi002.stdout:1586 write 0xbfad704 thru 0xbfbb69a (0xdf97 bytes) 2024-09-22T01:01:17.639 INFO:teuthology.orchestra.run.smithi002.stdout:1587 writesame 0x12e7a4 thru 0x12e833 data_size 0x4(0x90 bytes) 2024-09-22T01:01:17.641 INFO:teuthology.orchestra.run.smithi002.stdout:1588 write 0xa8c98c9 thru 0xa8d00ff (0x6837 bytes) 2024-09-22T01:01:17.649 INFO:teuthology.orchestra.run.smithi002.stdout:1589 write 0x1f18368 thru 0x1f19720 (0x13b9 bytes) 2024-09-22T01:01:17.655 INFO:teuthology.orchestra.run.smithi002.stdout:1590 write 0x134cf3f thru 0x1352c48 (0x5d0a bytes) 2024-09-22T01:01:17.664 INFO:teuthology.orchestra.run.smithi002.stdout:1592 write 0x893d124 thru 0x894bd40 (0xec1d bytes) 2024-09-22T01:01:17.675 INFO:teuthology.orchestra.run.smithi002.stdout:1593 punch from 0x401abc6 to 0x402258d, (0x79c7 bytes) 2024-09-22T01:01:17.676 INFO:teuthology.orchestra.run.smithi002.stdout:1594 punch from 0x4f85962 to 0x4f85ba9, (0x247 bytes) 2024-09-22T01:01:17.676 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.676 INFO:teuthology.orchestra.run.smithi002.stdout:1596 punch from 0x71fe554 to 0x7200e29, (0x28d5 bytes) 2024-09-22T01:01:17.679 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:17.676+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:17.693 INFO:teuthology.orchestra.run.smithi002.stdout:1597 trunc from 0xda78d00 to 0x4fa68b7 2024-09-22T01:01:17.714 INFO:teuthology.orchestra.run.smithi002.stdout:1600 read 0x33b4ef1 thru 0x33b85fc (0x370c bytes) 2024-09-22T01:01:17.714 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.714 INFO:teuthology.orchestra.run.smithi002.stdout:1602 punch from 0x47deb9 to 0x480c87, (0x2dce bytes) 2024-09-22T01:01:17.714 INFO:teuthology.orchestra.run.smithi002.stdout:1603 writesame 0x4436e02 thru 0x443d058 data_size 0x5(0x6257 bytes) 2024-09-22T01:01:17.721 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:17.721 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:17.721 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:17.721 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:17.723 INFO:teuthology.orchestra.run.smithi002.stdout:1604 trunc from 0x4fa68b7 to 0x33dae6f 2024-09-22T01:01:17.738 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:17.767 INFO:teuthology.orchestra.run.smithi002.stdout:1606 write 0xc320f58 thru 0xc329080 (0x8129 bytes) 2024-09-22T01:01:17.783 INFO:teuthology.orchestra.run.smithi002.stdout:1608 trunc from 0xc329081 to 0x6957589 2024-09-22T01:01:17.783 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:17.780+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:17.810 INFO:teuthology.orchestra.run.smithi002.stdout:1609 writesame 0x837c813 thru 0x838bf99 data_size 0x1(0xf787 bytes) 2024-09-22T01:01:17.811 INFO:teuthology.orchestra.run.smithi002.stdout:1611 punch from 0x3ee3928 to 0x3eeb035, (0x770d bytes) 2024-09-22T01:01:17.822 INFO:teuthology.orchestra.run.smithi002.stdout:1612 write 0x97f2ac9 thru 0x97f313e (0x676 bytes) 2024-09-22T01:01:17.829 INFO:teuthology.orchestra.run.smithi002.stdout:1613 punch from 0x3941c52 to 0x3942cdd, (0x108b bytes) 2024-09-22T01:01:17.829 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.829 INFO:teuthology.orchestra.run.smithi002.stdout:1616 read 0x81ba628 thru 0x81c1dd2 (0x77ab bytes) 2024-09-22T01:01:17.829 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.829 INFO:teuthology.orchestra.run.smithi002.stdout:1618 read 0x92118ac thru 0x9212026 (0x77b bytes) 2024-09-22T01:01:17.829 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.829 INFO:teuthology.orchestra.run.smithi002.stdout:1620 writesame 0x5cde35f thru 0x5cedbc1 data_size 0x1(0xf863 bytes) 2024-09-22T01:01:17.876 INFO:teuthology.orchestra.run.smithi002.stdout:1622 write 0x9e29bd1 thru 0x9e2a454 (0x884 bytes) 2024-09-22T01:01:17.881 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:17.881+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:17.883 INFO:teuthology.orchestra.run.smithi002.stdout:1623 punch from 0x3760299 to 0x3768aaa, (0x8811 bytes) 2024-09-22T01:01:17.883 INFO:teuthology.orchestra.run.smithi002.stdout:1624 write 0x37fd0a8 thru 0x38070e8 (0xa041 bytes) 2024-09-22T01:01:17.892 INFO:teuthology.orchestra.run.smithi002.stdout:1625 write 0x5c2be5 thru 0x5c770d (0x4b29 bytes) 2024-09-22T01:01:17.895 INFO:teuthology.orchestra.run.smithi002.stdout:1626 read 0x2f3d1fd thru 0x2f43916 (0x671a bytes) 2024-09-22T01:01:17.895 INFO:teuthology.orchestra.run.smithi002.stdout:1627 trunc from 0x9e2a455 to 0xa234b66 2024-09-22T01:01:17.907 INFO:teuthology.orchestra.run.smithi002.stdout:1628 write 0x6d9f90e thru 0x6da6572 (0x6c65 bytes) 2024-09-22T01:01:17.914 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:17.915 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:17.915 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:17.915 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:17.915 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:17.916 INFO:teuthology.orchestra.run.smithi002.stdout:1631 punch from 0x758c5e9 to 0x7596510, (0x9f27 bytes) 2024-09-22T01:01:17.916 INFO:teuthology.orchestra.run.smithi002.stdout:1632 read 0x3dc8576 thru 0x3dd2680 (0xa10b bytes) 2024-09-22T01:01:17.917 INFO:teuthology.orchestra.run.smithi002.stdout:1633 read 0x68eaa88 thru 0x68f5628 (0xaba1 bytes) 2024-09-22T01:01:17.917 INFO:teuthology.orchestra.run.smithi002.stdout:1634 write 0x3aa0293 thru 0x3aa8e0b (0x8b79 bytes) 2024-09-22T01:01:17.925 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.925 INFO:teuthology.orchestra.run.smithi002.stdout:1637 write 0x4bfd1c0 thru 0x4c0b929 (0xe76a bytes) 2024-09-22T01:01:17.931 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:17.934 INFO:teuthology.orchestra.run.smithi002.stdout:1638 write 0x521d8b0 thru 0x521e627 (0xd78 bytes) 2024-09-22T01:01:17.952 INFO:teuthology.orchestra.run.smithi002.stdout:1639 write 0xe044be5 thru 0xe044c2b (0x47 bytes) 2024-09-22T01:01:17.956 INFO:teuthology.orchestra.run.smithi002.stdout:1640 write 0x2042e33 thru 0x2045b2f (0x2cfd bytes) 2024-09-22T01:01:17.961 INFO:teuthology.orchestra.run.smithi002.stdout:1641 read 0xd271b3e thru 0xd27e985 (0xce48 bytes) 2024-09-22T01:01:17.961 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:17.962 INFO:teuthology.orchestra.run.smithi002.stdout:1643 write 0xa943642 thru 0xa9511e5 (0xdba4 bytes) 2024-09-22T01:01:17.980 INFO:teuthology.orchestra.run.smithi002.stdout:1644 writesame 0xe505228 thru 0xe507ce0 data_size 0x1(0x2ab9 bytes) 2024-09-22T01:01:17.980 INFO:teuthology.orchestra.run.smithi002.stdout:1646 read 0x7409a36 thru 0x74104d0 (0x6a9b bytes) 2024-09-22T01:01:17.980 INFO:teuthology.orchestra.run.smithi002.stdout:1647 punch from 0x727d987 to 0x728c25d, (0xe8d6 bytes) 2024-09-22T01:01:17.981 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:17.980+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:17.996 INFO:teuthology.orchestra.run.smithi002.stdout:1648 trunc from 0xe507ce1 to 0x4ae38a6 2024-09-22T01:01:18.016 INFO:teuthology.orchestra.run.smithi002.stdout:1650 writesame 0x3c43486 thru 0x3c4d899 data_size 0x2(0xa414 bytes) 2024-09-22T01:01:18.053 INFO:teuthology.orchestra.run.smithi002.stdout:1651 write 0xdc4ea91 thru 0xdc50a79 (0x1fe9 bytes) 2024-09-22T01:01:18.061 INFO:teuthology.orchestra.run.smithi002.stdout:1652 write 0x85f299 thru 0x86cb9f (0xd907 bytes) 2024-09-22T01:01:18.066 INFO:teuthology.orchestra.run.smithi002.stdout:1653 writesame 0x65ccbed thru 0x65dc9c1 data_size 0x1(0xfdd5 bytes) 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1182 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.013394655999999, 2024-09-22T01:01:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054296337, 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1182 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2399999999999999e-07 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061556+0000", 2024-09-22T01:01:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3700000000000002e-06 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061563+0000", 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0149999999999998e-06 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061568+0000", 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115682+0000", 2024-09-22T01:01:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054113513000000002 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115767+0000", 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4670000000000001e-05 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115850+0000", 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3264999999999996e-05 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1183 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.013372919999998, 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054497446999999997, 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1183 2024-09-22T01:01:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061576+0000", 2024-09-22T01:01:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.65e-07 2024-09-22T01:01:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061585+0000", 2024-09-22T01:01:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4960000000000002e-06 2024-09-22T01:01:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061593+0000", 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5340000000000002e-06 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061603+0000", 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0213e-05 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115905+0000", 2024-09-22T01:01:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054302187000000002 2024-09-22T01:01:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115985+0000", 2024-09-22T01:01:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9429000000000001e-05 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116073+0000", 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8222999999999999e-05 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1184 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.013337590999999, 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054590382999999999, 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1184 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061613+0000", 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061620+0000", 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.781e-06 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061625+0000", 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8999999999999997e-06 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116113+0000", 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054488414999999998 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116154+0000", 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0552000000000001e-05 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116201+0000", 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1185 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.013316321000001, 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054816747999999998, 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1185 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.34e-07 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061634+0000", 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.996e-06 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061641+0000", 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.455e-06 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061645+0000", 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6979999999999997e-06 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116274+0000", 2024-09-22T01:01:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054628659000000003 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116363+0000", 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8721000000000007e-05 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116449+0000", 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5785000000000006e-05 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1186 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.013296231999998, 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055030429999999998, 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1186 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061662+0000", 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1700000000000003e-06 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061669+0000", 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4109999999999999e-06 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061674+0000", 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999999e-06 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116496+0000", 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054821684000000002 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116585+0000", 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9276000000000004e-05 2024-09-22T01:01:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116682+0000", 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7386000000000006e-05 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1187 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.013266213000001, 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055229227999999998, 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1187 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061685+0000", 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.289e-06 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061691+0000", 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7710000000000001e-06 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061702+0000", 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0443e-05 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116727+0000", 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055024977000000003 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116811+0000", 2024-09-22T01:01:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4524000000000006e-05 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116911+0000", 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9802000000000004e-05 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1282 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.012289629000001, 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054493779999999999, 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1282 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062660+0000", 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6500000000000008e-07 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062664+0000", 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2080000000000002e-06 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062676+0000", 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.216e-05 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062684+0000", 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.576e-06 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116948+0000", 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054264616000000002 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117035+0000", 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7331999999999997e-05 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117152+0000", 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011692300000000001 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1283 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.012251766999999, 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054680980999999997, 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1283 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062697+0000", 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3900000000000005e-07 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062700+0000", 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062708+0000", 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8019999999999992e-06 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062713+0000", 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609999999999998e-06 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117187+0000", 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054473426999999998 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117270+0000", 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3353e-05 2024-09-22T01:01:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117377+0000", 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010757899999999999 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1284 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.012226025, 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054881757000000003, 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1284 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062723+0000", 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062725+0000", 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062732+0000", 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062737+0000", 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2580000000000003e-06 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117413+0000", 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054675834 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117498+0000", 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4753000000000007e-05 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117604+0000", 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106363 2024-09-22T01:01:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1285 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.012203955, 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055095997000000001, 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1285 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062745+0000", 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062747+0000", 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0760000000000001e-06 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062754+0000", 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.844e-06 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062759+0000", 2024-09-22T01:01:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9799999999999998e-06 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117639+0000", 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054880802999999999 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117723+0000", 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3746999999999996e-05 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117840+0000", 2024-09-22T01:01:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011718900000000001 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1286 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.012169706000002, 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055282537, 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1286 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062782+0000", 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.835e-06 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062790+0000", 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.055e-06 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062796+0000", 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117873+0000", 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055077470000000003 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117951+0000", 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118061+0000", 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110027 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1287 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.01214448, 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055478658, 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1287 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062806+0000", 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.142e-06 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062813+0000", 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.613e-06 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062818+0000", 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118096+0000", 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055277739999999999 2024-09-22T01:01:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118180+0000", 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4795000000000004e-05 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118282+0000", 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101974 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1288 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.012120487000001, 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055678514999999998, 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1288 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1199999999999998e-07 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062830+0000", 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.114e-06 2024-09-22T01:01:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062838+0000", 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8599999999999993e-06 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062843+0000", 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118315+0000", 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055471883999999999 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118402+0000", 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6992999999999999e-05 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118506+0000", 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104238 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1289 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.012087300000001, 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055873772000000002, 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1289 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063911+0000", 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010504500000000001 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063923+0000", 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1805e-05 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063947+0000", 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3519999999999998e-05 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063964+0000", 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6966e-05 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118539+0000", 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054574947999999998 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118631+0000", 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2238999999999996e-05 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118735+0000", 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103844 2024-09-22T01:01:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1290 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.010944473999999, 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054954785999999999, 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1290 2024-09-22T01:01:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064013+0000", 2024-09-22T01:01:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064020+0000", 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0310000000000001e-06 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064044+0000", 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4340000000000001e-05 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064081+0000", 2024-09-22T01:01:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6720000000000001e-05 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118769+0000", 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054687604000000001 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118852+0000", 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118959+0000", 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106969 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1291 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.010844200000001, 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055078496999999997, 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1291 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064105+0000", 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.268e-06 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064154+0000", 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9122999999999997e-05 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064174+0000", 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0057000000000001e-05 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064189+0000", 2024-09-22T01:01:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4083000000000001e-05 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118992+0000", 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054803604999999998 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119079+0000", 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7109999999999998e-05 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119183+0000", 2024-09-22T01:01:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103251 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1292 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.010737897999999, 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055189561999999998, 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1292 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065373+0000", 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001162419 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065382+0000", 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6979999999999993e-06 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065403+0000", 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0676e-05 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065418+0000", 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.468e-05 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119218+0000", 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053799877000000003 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119301+0000", 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2910000000000004e-05 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119400+0000", 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9302000000000006e-05 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1295 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.009378638000001, 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054395285000000002, 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1295 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065571+0000", 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065575+0000", 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065590+0000", 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.456e-05 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065616+0000", 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5935000000000002e-05 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119786+0000", 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054170124 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119862+0000", 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5833000000000004e-05 2024-09-22T01:01:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119965+0000", 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103281 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1296 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1327104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.009316331000001, 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054554510000000001, 2024-09-22T01:01:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1296 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066090+0000", 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045860600000000002 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066104+0000", 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3883000000000001e-05 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066125+0000", 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0392000000000001e-05 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066139+0000", 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3925e-05 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120000+0000", 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053860906 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120084+0000", 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4875000000000003e-05 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120186+0000", 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010192300000000001 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1297 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.008787270999999, 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054246075999999997, 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1297 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066728+0000", 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056672199999999997 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066733+0000", 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7400000000000001e-06 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066745+0000", 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.166e-05 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066753+0000", 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1620000000000008e-06 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120220+0000", 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05346687 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120303+0000", 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3354000000000002e-05 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120407+0000", 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103568 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:18.164 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:18.165 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:18.166 INFO:teuthology.orchestra.run.smithi002.stdout:1654 read 0x5a1ce7 thru 0x5a383b (0x1b55 bytes) 2024-09-22T01:01:18.166 INFO:teuthology.orchestra.run.smithi002.stdout:1655 writesame 0x88db5e4 thru 0x88e65d7 data_size 0x2(0xaff4 bytes) 2024-09-22T01:01:18.166 INFO:teuthology.orchestra.run.smithi002.stdout:1656 trunc from 0xdc50a7a to 0x584c84c 2024-09-22T01:01:18.167 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:18.164+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:18.176 INFO:teuthology.orchestra.run.smithi002.stdout:1659 write 0xaaf32f thru 0xabc133 (0xce05 bytes) 2024-09-22T01:01:18.206 INFO:teuthology.orchestra.run.smithi002.stdout:1660 writesame 0xa6bc7b8 thru 0xa6bd2dc data_size 0x3(0xb25 bytes) 2024-09-22T01:01:18.207 INFO:teuthology.orchestra.run.smithi002.stdout:1661 read 0x825a3c9 thru 0x825cd59 (0x2991 bytes) 2024-09-22T01:01:18.207 INFO:teuthology.orchestra.run.smithi002.stdout:1662 read 0x2e3aca3 thru 0x2e45785 (0xaae3 bytes) 2024-09-22T01:01:18.208 INFO:teuthology.orchestra.run.smithi002.stdout:1663 writesame 0x74b8fa6 thru 0x74c248d data_size 0x14(0x94e8 bytes) 2024-09-22T01:01:18.216 INFO:teuthology.orchestra.run.smithi002.stdout:1664 read 0x4bdfa57 thru 0x4beba92 (0xc03c bytes) 2024-09-22T01:01:18.216 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:18.216 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:18.231 INFO:teuthology.orchestra.run.smithi002.stdout:1667 write 0xb62040e thru 0xb62e6d1 (0xe2c4 bytes) 2024-09-22T01:01:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:1668 read 0x6aff6df thru 0x6b05db8 (0x66da bytes) 2024-09-22T01:01:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:1670 punch from 0x50a013a to 0x50a227e, (0x2144 bytes) 2024-09-22T01:01:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:1673 read 0x40bc7fc thru 0x40c7ad9 (0xb2de bytes) 2024-09-22T01:01:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:18.265 INFO:teuthology.orchestra.run.smithi002.stdout:1675 trunc from 0xb62e6d2 to 0x166dd59 2024-09-22T01:01:18.266 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:18.264+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:18.303 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:18.303 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:18.303 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:18.303 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:18.317 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:18.334 INFO:teuthology.orchestra.run.smithi002.stdout:1676 write 0xb4f77ad thru 0xb4ff0ea (0x793e bytes) 2024-09-22T01:01:18.367 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:18.364+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:18.370 INFO:teuthology.orchestra.run.smithi002.stdout:1678 trunc from 0xb4ff0eb to 0x1e1733 2024-09-22T01:01:18.394 INFO:teuthology.orchestra.run.smithi002.stdout:1679 read 0xe7e2b thru 0xf72a3 (0xf479 bytes) 2024-09-22T01:01:18.432 INFO:teuthology.orchestra.run.smithi002.stdout:1680 trunc from 0x1e1733 to 0xed8e0d5 2024-09-22T01:01:18.443 INFO:teuthology.orchestra.run.smithi002.stdout:1681 write 0xa576983 thru 0xa57992d (0x2fab bytes) 2024-09-22T01:01:18.455 INFO:teuthology.orchestra.run.smithi002.stdout:1683 trunc from 0xed8e0d5 to 0xe1386b2 2024-09-22T01:01:18.467 INFO:teuthology.orchestra.run.smithi002.stdout:1684 read 0x9682479 thru 0x96886cf (0x6257 bytes) 2024-09-22T01:01:18.467 INFO:teuthology.orchestra.run.smithi002.stdout:1685 write 0x26a7a7f thru 0x26b0048 (0x85ca bytes) 2024-09-22T01:01:18.468 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:18.464+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:18.478 INFO:teuthology.orchestra.run.smithi002.stdout:1686 punch from 0x4a089a0 to 0x4a104ac, (0x7b0c bytes) 2024-09-22T01:01:18.478 INFO:teuthology.orchestra.run.smithi002.stdout:1687 write 0xd547e9b thru 0xd54edc7 (0x6f2d bytes) 2024-09-22T01:01:18.486 INFO:teuthology.orchestra.run.smithi002.stdout:1688 writesame 0x1e0f961 thru 0x1e1c1b7 data_size 0x1(0xc857 bytes) 2024-09-22T01:01:18.488 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:18.488 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:18.488 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:18.488 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:18.489 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:18.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:18.517 INFO:teuthology.orchestra.run.smithi002.stdout:1690 punch from 0x182380c to 0x182af56, (0x774a bytes) 2024-09-22T01:01:18.540 INFO:teuthology.orchestra.run.smithi002.stdout:1691 trunc from 0xe1386b2 to 0x59a7aa6 2024-09-22T01:01:18.560 INFO:teuthology.orchestra.run.smithi002.stdout:1692 read 0x4c401e thru 0x4d073b (0xc71e bytes) 2024-09-22T01:01:18.562 INFO:teuthology.orchestra.run.smithi002.stdout:1694 writesame 0x27456ef thru 0x274f358 data_size 0x1(0x9c6a bytes) 2024-09-22T01:01:18.568 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:18.564+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:18.606 INFO:teuthology.orchestra.run.smithi002.stdout:1696 punch from 0x2a78ac2 to 0x2a83124, (0xa662 bytes) 2024-09-22T01:01:18.606 INFO:teuthology.orchestra.run.smithi002.stdout:1697 write 0x4104523 thru 0x4106ed4 (0x29b2 bytes) 2024-09-22T01:01:18.625 INFO:teuthology.orchestra.run.smithi002.stdout:1699 trunc from 0x59a7aa6 to 0x8cc8c 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6684322740000002, 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.638 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.668379751, 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.642 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6683274299999999, 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.646 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6681616560000001, 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6681332279999999, 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.668105419, 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:01:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:01:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.668077314, 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.668049772, 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:01:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:01:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6680221240000002, 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6679949789999999, 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6679670199999999, 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:01:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:01:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6679392050000001, 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:01:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.66791226, 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:01:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:01:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.667884559, 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:01:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:01:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:01:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.667770135, 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:01:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6677307770000001, 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.6677006219999999, 2024-09-22T01:01:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.667671364, 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:01:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:01:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.667643488, 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:01:18.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:01:18.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.667614114, 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:18.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:18.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:18.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:18.716 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:18.716 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:18.716 INFO:teuthology.orchestra.run.smithi002.stdout:1702 trunc from 0x8cc8c to 0x23dece5 2024-09-22T01:01:18.717 INFO:teuthology.orchestra.run.smithi002.stdout:1703 write 0xe203d37 thru 0xe20a26f (0x6539 bytes) 2024-09-22T01:01:18.717 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:18.718 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:18.716+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:18.725 INFO:teuthology.orchestra.run.smithi002.stdout:1704 read 0xd84f37f thru 0xd852b98 (0x381a bytes) 2024-09-22T01:01:18.725 INFO:teuthology.orchestra.run.smithi002.stdout:1705 punch from 0x9b88e16 to 0x9b93a12, (0xabfc bytes) 2024-09-22T01:01:18.725 INFO:teuthology.orchestra.run.smithi002.stdout:1706 write 0x2867c20 thru 0x287551d (0xd8fe bytes) 2024-09-22T01:01:18.733 INFO:teuthology.orchestra.run.smithi002.stdout:1707 punch from 0x23ef397 to 0x23f5113, (0x5d7c bytes) 2024-09-22T01:01:18.734 INFO:teuthology.orchestra.run.smithi002.stdout:1710 punch from 0x880d04f to 0x881c52e, (0xf4df bytes) 2024-09-22T01:01:18.734 INFO:teuthology.orchestra.run.smithi002.stdout:1711 writesame 0x22a7176 thru 0x22ad896 data_size 0x1(0x6721 bytes) 2024-09-22T01:01:18.755 INFO:teuthology.orchestra.run.smithi002.stdout:1714 write 0x6a062b2 thru 0x6a15d8e (0xfadd bytes) 2024-09-22T01:01:18.761 INFO:teuthology.orchestra.run.smithi002.stdout:1715 write 0x2087cfc thru 0x2091b16 (0x9e1b bytes) 2024-09-22T01:01:18.767 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:18.769 INFO:teuthology.orchestra.run.smithi002.stdout:1716 write 0x3c61258 thru 0x3c6baa0 (0xa849 bytes) 2024-09-22T01:01:18.776 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:18.776 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:18.791 INFO:teuthology.orchestra.run.smithi002.stdout:1719 trunc from 0xe20a270 to 0x86a2861 2024-09-22T01:01:18.818 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:18.816+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:18.829 INFO:teuthology.orchestra.run.smithi002.stdout:1720 write 0xed141cb thru 0xed19a91 (0x58c7 bytes) 2024-09-22T01:01:18.848 INFO:teuthology.orchestra.run.smithi002.stdout:1722 trunc from 0xed19a92 to 0x84e01b2 2024-09-22T01:01:18.852 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:18.852 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:18.868 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:1723 punch from 0x35e9cce to 0x35f749d, (0xd7cf bytes) 2024-09-22T01:01:18.882 INFO:teuthology.orchestra.run.smithi002.stdout:1724 writesame 0x8a84d0f thru 0x8a85926 data_size 0xc(0xc18 bytes) 2024-09-22T01:01:18.882 INFO:teuthology.orchestra.run.smithi002.stdout:1725 read 0xe4dd1c thru 0xe5a96e (0xcc53 bytes) 2024-09-22T01:01:18.883 INFO:teuthology.orchestra.run.smithi002.stdout:1729 write 0x7c538d2 thru 0x7c54e74 (0x15a3 bytes) 2024-09-22T01:01:18.900 INFO:teuthology.orchestra.run.smithi002.stdout:1730 trunc from 0x8a85927 to 0x3605592 2024-09-22T01:01:18.911 INFO:teuthology.orchestra.run.smithi002.stdout:1731 read 0x101af9b thru 0x101b383 (0x3e9 bytes) 2024-09-22T01:01:18.918 INFO:teuthology.orchestra.run.smithi002.stdout:1732 trunc from 0x3605592 to 0x7b48581 2024-09-22T01:01:18.918 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:18.916+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:1733 read 0x25ed29d thru 0x25f978b (0xc4ef bytes) 2024-09-22T01:01:18.924 INFO:teuthology.orchestra.run.smithi002.stdout:1736 writesame 0x10f0acb thru 0x10fbb4d data_size 0x1(0xb083 bytes) 2024-09-22T01:01:18.933 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:18.934 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:18.945 INFO:teuthology.orchestra.run.smithi002.stdout:1737 punch from 0x1899083 to 0x18a8a0b, (0xf988 bytes) 2024-09-22T01:01:18.945 INFO:teuthology.orchestra.run.smithi002.stdout:1738 clone 14 order 22 su 16384 sc 10 2024-09-22T01:01:18.950 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:18.967 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:01:18.967 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:18.968 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:18.984 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:18.986 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:18.995 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:18.995 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:18.995 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:18.995 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:18.995 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:19.004 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:19.005 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:19.005 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:19.007 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-22T01:01:19.007 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-22T01:01:19.007 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-22T01:01:19.007 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-22T01:01:19.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:19.009 INFO:tasks.ceph.osd.5:Started 2024-09-22T01:01:19.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:19.025 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:19.025 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:19.043 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:19.044 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:19.047 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:19.048 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:19.052 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:19.052+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:19.120 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:19.120+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:19.162 INFO:teuthology.orchestra.run.smithi119.stderr:no valid command found; 10 closest matches: 2024-09-22T01:01:19.162 INFO:teuthology.orchestra.run.smithi119.stderr:0 2024-09-22T01:01:19.162 INFO:teuthology.orchestra.run.smithi119.stderr:1 2024-09-22T01:01:19.162 INFO:teuthology.orchestra.run.smithi119.stderr:2 2024-09-22T01:01:19.162 INFO:teuthology.orchestra.run.smithi119.stderr:abort 2024-09-22T01:01:19.162 INFO:teuthology.orchestra.run.smithi119.stderr:assert 2024-09-22T01:01:19.162 INFO:teuthology.orchestra.run.smithi119.stderr:config diff 2024-09-22T01:01:19.163 INFO:teuthology.orchestra.run.smithi119.stderr:config diff get 2024-09-22T01:01:19.163 INFO:teuthology.orchestra.run.smithi119.stderr:config get 2024-09-22T01:01:19.163 INFO:teuthology.orchestra.run.smithi119.stderr:config help [] 2024-09-22T01:01:19.163 INFO:teuthology.orchestra.run.smithi119.stderr:config set ... 2024-09-22T01:01:19.163 INFO:teuthology.orchestra.run.smithi119.stderr:admin_socket: invalid command 2024-09-22T01:01:19.165 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T01:01:19.165 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-22T01:01:19.169 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:19.169 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:19.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:19.228 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:19.228+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:19.322 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:19.320+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:19.348 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:19.348 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:19.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:19.423 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:19.420+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:19.487 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:19.487 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:19.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:19.523 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:19.520+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:19.622 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:19.622 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:19.625 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:19.624+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:19.636 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:19.725 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:19.724+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:19.772 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:19.773 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:19.787 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:19.829 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:19.828+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:19.901 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:01:19.926 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:19.924+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:19.972 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:19.972 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:19.987 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:20.028 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone13 from 0x7b48581 (overlap 0x8cc8c) to 0x0 2024-09-22T01:01:20.028 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:20.024+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:20.116 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:20.116 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:20.127 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:20.124+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:20.131 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:20.165 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-22T01:01:20.167 INFO:teuthology.orchestra.run.smithi002.stdout:1740 punch from 0x17fa64 to 0x185a7a, (0x6016 bytes) 2024-09-22T01:01:20.174 INFO:teuthology.orchestra.run.smithi002.stdout:1741 read 0x7286c72 thru 0x728f6d8 (0x8a67 bytes) 2024-09-22T01:01:20.174 INFO:teuthology.orchestra.run.smithi002.stdout:1742 read 0x34b2272 thru 0x34c01e3 (0xdf72 bytes) 2024-09-22T01:01:20.174 INFO:teuthology.orchestra.run.smithi002.stdout:1743 read 0x3df429e thru 0x3e021c6 (0xdf29 bytes) 2024-09-22T01:01:20.174 INFO:teuthology.orchestra.run.smithi002.stdout:1745 write 0x468ed45 thru 0x469a694 (0xb950 bytes) 2024-09-22T01:01:20.207 INFO:teuthology.orchestra.run.smithi002.stdout:1746 write 0x8680253 thru 0x8680f84 (0xd32 bytes) 2024-09-22T01:01:20.214 INFO:teuthology.orchestra.run.smithi002.stdout:1747 read 0x18d57c1 thru 0x18e4087 (0xe8c7 bytes) 2024-09-22T01:01:20.214 INFO:teuthology.orchestra.run.smithi002.stdout:1748 writesame 0x3d8329f thru 0x3d8bda8 data_size 0x25(0x8b0a bytes) 2024-09-22T01:01:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:20.228 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:20.228+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:20.229 INFO:teuthology.orchestra.run.smithi002.stdout:1750 write 0x95f2a33 thru 0x95f3e46 (0x1414 bytes) 2024-09-22T01:01:20.236 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:1753 writesame 0xcc1fde1 thru 0xcc2f291 data_size 0x1(0xf4b1 bytes) 2024-09-22T01:01:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:1754 punch from 0x3769734 to 0x37741ff, (0xaacb bytes) 2024-09-22T01:01:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:1755 punch from 0xc4484b1 to 0xc4565e8, (0xe137 bytes) 2024-09-22T01:01:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:1756 clone 15 order 21 su 32768 sc 15 2024-09-22T01:01:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:20.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:20.329 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:20.328+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:20.430 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:20.428+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:20.475 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:20.475 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:20.489 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:20.533 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:20.532+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:20.632 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:20.628+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:20.662 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:20.662 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:20.676 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:20.761 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:20.760+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:20.833 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:20.832+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:20.845 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:20.846 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:20.860 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:20.945 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:20.944+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:21.033 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:21.033 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:21.036 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:21.032+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:21.048 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:21.049 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:21.076 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone14 from 0xcc2f292 (overlap 0x7b48581) to 0x6ef44f4 2024-09-22T01:01:21.138 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:21.136+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:21.138 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:21.136+0000 7fa3818427c0 -1 Falling back to public interface 2024-09-22T01:01:21.211 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:21.211 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:21.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:21.245 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:21.245 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:21.246 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:21.244+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:21.264 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:21.264 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:21.284 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:21.285 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:21.302 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:21.302 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:21.310 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-22T01:01:21.311 INFO:teuthology.orchestra.run.smithi002.stdout:1758 write 0x354dbd6 thru 0x35507c9 (0x2bf4 bytes) 2024-09-22T01:01:21.324 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:21.330 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:21.330 INFO:teuthology.orchestra.run.smithi002.stdout:1762 writesame 0x2d790a0 thru 0x2d84e0a data_size 0x1(0xbd6b bytes) 2024-09-22T01:01:21.338 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:21.336+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:21.347 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:21.347 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:21.353 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:21.353 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:21.353 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:21.353 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:21.354 INFO:teuthology.orchestra.run.smithi002.stdout:1763 writesame 0x9475035 thru 0x94754a6 data_size 0x1(0x472 bytes) 2024-09-22T01:01:21.369 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:21.373 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:21.373 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:21.377 INFO:teuthology.orchestra.run.smithi002.stdout:1764 trunc from 0xcc2f292 to 0x361f640 2024-09-22T01:01:21.379 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:21.440 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:21.440+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:21.470 INFO:teuthology.orchestra.run.smithi002.stdout:1765 read 0x1d87a72 thru 0x1d92fb8 (0xb547 bytes) 2024-09-22T01:01:21.470 INFO:teuthology.orchestra.run.smithi002.stdout:1766 punch from 0x19d12e3 to 0x19daf06, (0x9c23 bytes) 2024-09-22T01:01:21.483 INFO:teuthology.orchestra.run.smithi002.stdout:1767 trunc from 0x361f640 to 0xe8c9f77 2024-09-22T01:01:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:1769 read 0x9031fb thru 0x90c2b5 (0x90bb bytes) 2024-09-22T01:01:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:1772 read 0x34e2418 thru 0x34e8bd9 (0x67c2 bytes) 2024-09-22T01:01:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:1774 read 0x581fcd7 thru 0x582d3e2 (0xd70c bytes) 2024-09-22T01:01:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:1775 write 0xda8f44b thru 0xda96221 (0x6dd7 bytes) 2024-09-22T01:01:21.497 INFO:teuthology.orchestra.run.smithi002.stdout:1776 read 0x2070b8d thru 0x2079541 (0x89b5 bytes) 2024-09-22T01:01:21.498 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:21.498 INFO:teuthology.orchestra.run.smithi002.stdout:1778 read 0x5d6b017 thru 0x5d776df (0xc6c9 bytes) 2024-09-22T01:01:21.498 INFO:teuthology.orchestra.run.smithi002.stdout:1779 write 0x4f7714a thru 0x4f82b3c (0xb9f3 bytes) 2024-09-22T01:01:21.504 INFO:teuthology.orchestra.run.smithi002.stdout:1780 read 0x94da8cb thru 0x94dd5fe (0x2d34 bytes) 2024-09-22T01:01:21.504 INFO:teuthology.orchestra.run.smithi002.stdout:1781 write 0xd45bc21 thru 0xd467e87 (0xc267 bytes) 2024-09-22T01:01:21.509 INFO:teuthology.orchestra.run.smithi002.stdout:1782 clone 16 order 24 su 2097152 sc 5 2024-09-22T01:01:21.541 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:21.540+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:21.547 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:21.547 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:21.547 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:21.547 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:21.547 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:21.560 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:21.641 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:21.640+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:21.678 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1182 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.615822802, 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054296337, 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1182 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061554+0000", 2024-09-22T01:01:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2399999999999999e-07 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061556+0000", 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3700000000000002e-06 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061563+0000", 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0149999999999998e-06 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061568+0000", 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115682+0000", 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054113513000000002 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115767+0000", 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4670000000000001e-05 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115850+0000", 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3264999999999996e-05 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1183 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.615801066, 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054497446999999997, 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1183 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061575+0000", 2024-09-22T01:01:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061576+0000", 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.65e-07 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061585+0000", 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4960000000000002e-06 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061593+0000", 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5340000000000002e-06 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061603+0000", 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0213e-05 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115905+0000", 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054302187000000002 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.115985+0000", 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9429000000000001e-05 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116073+0000", 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8222999999999999e-05 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1184 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.615765737, 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054590382999999999, 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1184 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061611+0000", 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061613+0000", 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061620+0000", 2024-09-22T01:01:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.781e-06 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061625+0000", 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8999999999999997e-06 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116113+0000", 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054488414999999998 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116154+0000", 2024-09-22T01:01:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0552000000000001e-05 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116201+0000", 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1185 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.615744466999999, 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054816747999999998, 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1185 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061632+0000", 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.34e-07 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061634+0000", 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.996e-06 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061641+0000", 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.455e-06 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061645+0000", 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6979999999999997e-06 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116274+0000", 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054628659000000003 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116363+0000", 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8721000000000007e-05 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116449+0000", 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5785000000000006e-05 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1186 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.615724377999999, 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055030429999999998, 2024-09-22T01:01:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1186 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061652+0000", 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061662+0000", 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1700000000000003e-06 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061669+0000", 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4109999999999999e-06 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061674+0000", 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999999e-06 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116496+0000", 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054821684000000002 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116585+0000", 2024-09-22T01:01:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9276000000000004e-05 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116682+0000", 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7386000000000006e-05 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1187 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.615694358999999, 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055229227999999998, 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1187 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061682+0000", 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061685+0000", 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.289e-06 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061691+0000", 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7710000000000001e-06 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.061702+0000", 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0443e-05 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116727+0000", 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055024977000000003 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116811+0000", 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4524000000000006e-05 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116911+0000", 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9802000000000004e-05 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1282 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.614717774999999, 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054493779999999999, 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1282 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062659+0000", 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062660+0000", 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6500000000000008e-07 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062664+0000", 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2080000000000002e-06 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062676+0000", 2024-09-22T01:01:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.216e-05 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062684+0000", 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.576e-06 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.116948+0000", 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054264616000000002 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117035+0000", 2024-09-22T01:01:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7331999999999997e-05 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117152+0000", 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011692300000000001 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1283 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.614679913, 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054680980999999997, 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1283 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062696+0000", 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062697+0000", 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3900000000000005e-07 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062700+0000", 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062708+0000", 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8019999999999992e-06 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062713+0000", 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609999999999998e-06 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117187+0000", 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054473426999999998 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117270+0000", 2024-09-22T01:01:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3353e-05 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117377+0000", 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010757899999999999 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1284 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.614654171000002, 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054881757000000003, 2024-09-22T01:01:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1284 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062722+0000", 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062723+0000", 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062725+0000", 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062732+0000", 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062737+0000", 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2580000000000003e-06 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117413+0000", 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054675834 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117498+0000", 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4753000000000007e-05 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117604+0000", 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106363 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1285 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.614632101000002, 2024-09-22T01:01:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055095997000000001, 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1285 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062744+0000", 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062745+0000", 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062747+0000", 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0760000000000001e-06 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062754+0000", 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.844e-06 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062759+0000", 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9799999999999998e-06 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117639+0000", 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054880802999999999 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117723+0000", 2024-09-22T01:01:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3746999999999996e-05 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117840+0000", 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011718900000000001 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1286 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.614597851999999, 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055282537, 2024-09-22T01:01:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1286 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062779+0000", 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062782+0000", 2024-09-22T01:01:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.835e-06 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062790+0000", 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.055e-06 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062796+0000", 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117873+0000", 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055077470000000003 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.117951+0000", 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118061+0000", 2024-09-22T01:01:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110027 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1287 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.614572626000001, 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055478658, 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1287 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062804+0000", 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062806+0000", 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.142e-06 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062813+0000", 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.613e-06 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062818+0000", 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118096+0000", 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055277739999999999 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118180+0000", 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4795000000000004e-05 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118282+0000", 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101974 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1288 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.614548632999998, 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055678514999999998, 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1288 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062828+0000", 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1199999999999998e-07 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062830+0000", 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.114e-06 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062838+0000", 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8599999999999993e-06 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062843+0000", 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118315+0000", 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055471883999999999 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118402+0000", 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6992999999999999e-05 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118506+0000", 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104238 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1289 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.614515445999999, 2024-09-22T01:01:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055873772000000002, 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1289 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.062861+0000", 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063911+0000", 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010504500000000001 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063923+0000", 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1805e-05 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063947+0000", 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3519999999999998e-05 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.063964+0000", 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6966e-05 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118539+0000", 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054574947999999998 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118631+0000", 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2238999999999996e-05 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118735+0000", 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103844 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1290 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.61337262, 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054954785999999999, 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1290 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064004+0000", 2024-09-22T01:01:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064013+0000", 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064020+0000", 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0310000000000001e-06 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064044+0000", 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4340000000000001e-05 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064081+0000", 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6720000000000001e-05 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118769+0000", 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054687604000000001 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118852+0000", 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118959+0000", 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106969 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1291 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.613272345999999, 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055078496999999997, 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1291 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064104+0000", 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064105+0000", 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.268e-06 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064154+0000", 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9122999999999997e-05 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064174+0000", 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0057000000000001e-05 2024-09-22T01:01:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064189+0000", 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4083000000000001e-05 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.118992+0000", 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054803604999999998 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119079+0000", 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7109999999999998e-05 2024-09-22T01:01:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119183+0000", 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103251 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1292 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.613166044, 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055189561999999998, 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1292 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.064210+0000", 2024-09-22T01:01:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065373+0000", 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001162419 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065382+0000", 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6979999999999993e-06 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065403+0000", 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0676e-05 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065418+0000", 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.468e-05 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119218+0000", 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053799877000000003 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119301+0000", 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2910000000000004e-05 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119400+0000", 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9302000000000006e-05 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1295 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.611806783999999, 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054395285000000002, 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1295 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065570+0000", 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065571+0000", 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065575+0000", 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065590+0000", 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.456e-05 2024-09-22T01:01:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065616+0000", 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5935000000000002e-05 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119786+0000", 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054170124 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119862+0000", 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5833000000000004e-05 2024-09-22T01:01:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.119965+0000", 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103281 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1296 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1327104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.611744476999998, 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054554510000000001, 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1296 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.065632+0000", 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066090+0000", 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045860600000000002 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066104+0000", 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3883000000000001e-05 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066125+0000", 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0392000000000001e-05 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066139+0000", 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3925e-05 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120000+0000", 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053860906 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120084+0000", 2024-09-22T01:01:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4875000000000003e-05 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120186+0000", 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010192300000000001 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1297 3.18 3:1bf0d7ea:::rbd_data.117cf538ee01.000000000000001d:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.611215417, 2024-09-22T01:01:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054246075999999997, 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1297 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066161+0000", 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066728+0000", 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056672199999999997 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066733+0000", 2024-09-22T01:01:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7400000000000001e-06 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066745+0000", 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.166e-05 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.066753+0000", 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1620000000000008e-06 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120220+0000", 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05346687 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120303+0000", 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3354000000000002e-05 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.120407+0000", 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103568 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:21.761 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:21.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:21.763 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:21.760+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:21.864 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:21.860+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:21.889 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:21.890 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:21.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:21.927 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:01:21.973 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:21.972+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:22.030 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:22.030 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:22.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:22.064 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:22.060+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:22.093 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone15 2024-09-22T01:01:22.165 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:22.164+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:22.184 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2146259959999997, 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2145734729999997, 2024-09-22T01:01:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:01:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:01:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2145211519999997, 2024-09-22T01:01:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2143553779999996, 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2143269500000002, 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:01:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:01:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2142991409999997, 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2142710360000004, 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:01:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:01:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2142434939999998, 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:01:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2142158460000001, 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:01:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:01:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2141887010000003, 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:01:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2141607419999998, 2024-09-22T01:01:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2141329269999996, 2024-09-22T01:01:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2141059820000004, 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:01:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2140782809999999, 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:01:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2139638570000004, 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:01:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.213924499, 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2138943439999998, 2024-09-22T01:01:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2138650860000002, 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2138372100000003, 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.213807836, 2024-09-22T01:01:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:01:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:01:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.260 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:22.261 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:22.265 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:22.264+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:22.367 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:22.363+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:22.436 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:22.437 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:22.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:22.468 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:22.463+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:22.569 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:22.567+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:22.574 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:22.575 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:22.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:22.669 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:22.667+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:22.684 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-22T01:01:22.716 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2955 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336647066000001, 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044281583999999999, 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2955 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378127+0000", 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378128+0000", 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378131+0000", 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.864e-06 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378137+0000", 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8020000000000003e-06 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378141+0000", 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422313+0000", 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044172553000000003 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422359+0000", 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6146000000000002e-05 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422409+0000", 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9576000000000001e-05 2024-09-22T01:01:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2956 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3284992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336617378, 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044364010000000002, 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2956 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378159+0000", 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8679999999999999e-06 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378164+0000", 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2499999999999997e-06 2024-09-22T01:01:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378169+0000", 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3900000000000003e-06 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422438+0000", 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044268764000000002 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422478+0000", 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.066e-05 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422521+0000", 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2741999999999998e-05 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2957 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336600657000002, 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044474276, 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2957 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1299999999999999e-07 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378175+0000", 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3039999999999999e-06 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378179+0000", 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.388e-06 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378188+0000", 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0569999999999999e-06 2024-09-22T01:01:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422550+0000", 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044362523000000001 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422597+0000", 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6493999999999999e-05 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422648+0000", 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1297e-05 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2958 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336580610999999, 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044568617999999997, 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2958 2024-09-22T01:01:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.75e-07 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378196+0000", 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4959999999999999e-06 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378199+0000", 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7969999999999998e-06 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378202+0000", 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.65e-06 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422678+0000", 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044475564000000002 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422718+0000", 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0825000000000003e-05 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422762+0000", 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3810999999999998e-05 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2959 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336568354000001, 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044681193000000001, 2024-09-22T01:01:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2959 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378207+0000", 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.316e-06 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378211+0000", 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378213+0000", 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.588e-06 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422794+0000", 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044580928999999998 2024-09-22T01:01:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422840+0000", 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5231000000000001e-05 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422887+0000", 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7543999999999997e-05 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2960 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336557309, 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044780915999999997, 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2960 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378218+0000", 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.032e-06 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378222+0000", 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4690000000000002e-06 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378224+0000", 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5040000000000001e-06 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422915+0000", 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044691036000000003 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422956+0000", 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0881999999999999e-05 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422998+0000", 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1782000000000001e-05 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2961 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336546687999999, 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891911, 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2961 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378229+0000", 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.071e-06 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378232+0000", 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378235+0000", 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4609999999999998e-06 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423029+0000", 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044794498000000002 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423072+0000", 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2796000000000002e-05 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423120+0000", 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7574999999999998e-05 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2962 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336536157000001, 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044991058, 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2962 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378239+0000", 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378243+0000", 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3459999999999998e-06 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378245+0000", 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6340000000000001e-06 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423149+0000", 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044903169999999999 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423188+0000", 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8974999999999999e-05 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423229+0000", 2024-09-22T01:01:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1655e-05 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2963 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336525545000001, 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104404000000001, 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378250+0000", 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.001e-06 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378255+0000", 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7949999999999998e-06 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378264+0000", 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6129999999999993e-06 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423259+0000", 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044994348000000003 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423302+0000", 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3733000000000003e-05 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423353+0000", 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0689000000000002e-05 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2964 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336505256999999, 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045200033000000001, 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2964 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378275+0000", 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3850000000000001e-06 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378279+0000", 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378281+0000", 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5129999999999999e-06 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423382+0000", 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045100385 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423422+0000", 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0265999999999997e-05 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423469+0000", 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7110999999999997e-05 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2965 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336482749000002, 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045300746000000003, 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2965 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6899999999999999e-07 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378294+0000", 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.951e-06 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378298+0000", 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1239999999999998e-06 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378301+0000", 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.779e-06 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423497+0000", 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045196473000000001 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423544+0000", 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6622000000000002e-05 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423592+0000", 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8528e-05 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2966 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336469351000002, 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045402558000000003, 2024-09-22T01:01:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2966 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378306+0000", 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2869999999999999e-06 2024-09-22T01:01:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378310+0000", 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.258e-06 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378312+0000", 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7810000000000001e-06 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423622+0000", 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045309464000000001 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423663+0000", 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1310999999999998e-05 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423708+0000", 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4248000000000001e-05 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3013 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336458254, 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045530159000000001, 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3013 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378317+0000", 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378321+0000", 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3110000000000001e-06 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378323+0000", 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5600000000000001e-06 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423737+0000", 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045414035999999998 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423781+0000", 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3279999999999999e-05 2024-09-22T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423846+0000", 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5634000000000002e-05 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3014 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336447595999999, 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045653157, 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3014 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378715+0000", 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038801599999999998 2024-09-22T01:01:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378719+0000", 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4039999999999997e-06 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378731+0000", 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1755e-05 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378739+0000", 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.712e-06 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423876+0000", 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045137505000000001 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423918+0000", 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2339999999999998e-05 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423980+0000", 2024-09-22T01:01:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1425000000000002e-05 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3015 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.336020221999998, 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045369596999999998, 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3015 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378755+0000", 2024-09-22T01:01:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0100000000000003e-07 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378759+0000", 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378766+0000", 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.029e-06 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378769+0000", 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7189999999999999e-06 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424013+0000", 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045243798000000002 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424059+0000", 2024-09-22T01:01:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5725e-05 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424124+0000", 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4851999999999995e-05 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3016 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.335998823000001, 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045478877000000001, 2024-09-22T01:01:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3016 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379160+0000", 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038423800000000002 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379164+0000", 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379173+0000", 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.439e-06 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379178+0000", 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3909999999999997e-06 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424155+0000", 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044976781 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424198+0000", 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769000000000003e-05 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424254+0000", 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6641000000000002e-05 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3017 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.335581783999999, 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045202985000000001, 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3017 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9400000000000005e-07 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379198+0000", 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379215+0000", 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6863000000000001e-05 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379219+0000", 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0749999999999999e-06 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424286+0000", 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045067013000000003 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424332+0000", 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5531000000000002e-05 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424395+0000", 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3740999999999998e-05 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3018 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.335546031, 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045293472000000001, 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3018 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379430+0000", 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020146699999999999 2024-09-22T01:01:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379435+0000", 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379443+0000", 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4819999999999994e-06 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379461+0000", 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.753e-05 2024-09-22T01:01:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424424+0000", 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044962716999999999 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424464+0000", 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0824000000000001e-05 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424522+0000", 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7383999999999998e-05 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3019 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.335303775, 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045194026999999998, 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3019 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381690+0000", 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022191849999999998 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381696+0000", 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3860000000000004e-06 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381705+0000", 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2480000000000002e-06 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381712+0000", 2024-09-22T01:01:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.579e-06 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424554+0000", 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042842327999999999 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424601+0000", 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6697999999999999e-05 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424665+0000", 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3602999999999994e-05 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:8990 3.a 3:51e071dd:::rbd_data.117cc87baa96.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.533757875999999, 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048973669999999997, 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 8990 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181019+0000", 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4889999999999998e-06 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181038+0000", 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9097e-05 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181066+0000", 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7514000000000001e-05 2024-09-22T01:01:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181078+0000", 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2626000000000001e-05 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181196+0000", 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118208 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181388+0000", 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019203100000000001 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.188674+0000", 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072856120000000003 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229668+0000", 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040994282 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229764+0000", 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6036e-05 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229859+0000", 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5088999999999997e-05 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229990+0000", 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130686 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:22.803 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:22.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:22.810 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:22.799+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:22.905 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:22.903+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:22.924 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:22.924 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:22.924 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:22.924 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:01:22.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:23.006 INFO:teuthology.orchestra.run.smithi002.stdout:1785 read 0x9369ce7 thru 0x936c246 (0x2560 bytes) 2024-09-22T01:01:23.006 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:23.004+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:23.009 INFO:teuthology.orchestra.run.smithi002.stdout:1787 write 0xdf660d1 thru 0xdf6c104 (0x6034 bytes) 2024-09-22T01:01:23.026 INFO:teuthology.orchestra.run.smithi002.stdout:1788 punch from 0x878e667 to 0x879ae1c, (0xc7b5 bytes) 2024-09-22T01:01:23.026 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:23.026 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:23.026 INFO:teuthology.orchestra.run.smithi002.stdout:1791 write 0xcf31bd6 thru 0xcf3b67f (0x9aaa bytes) 2024-09-22T01:01:23.034 INFO:teuthology.orchestra.run.smithi002.stdout:1792 writesame 0x4b7dd91 thru 0x4b8624b data_size 0x1(0x84bb bytes) 2024-09-22T01:01:23.065 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:23.066 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:23.066 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:23.066 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:23.066 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:23.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:23.087 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:23.087 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:23.087 INFO:teuthology.orchestra.run.smithi002.stdout:1795 write 0x7653b2c thru 0x7656c5a (0x312f bytes) 2024-09-22T01:01:23.108 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:23.103+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:23.117 INFO:teuthology.orchestra.run.smithi002.stdout:1796 trunc from 0xe8c9f77 to 0x389042f 2024-09-22T01:01:23.208 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:23.203+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:23.232 INFO:teuthology.orchestra.run.smithi002.stdout:1797 writesame 0xea95168 thru 0xea9a867 data_size 0x20(0x5700 bytes) 2024-09-22T01:01:23.232 INFO:teuthology.orchestra.run.smithi002.stdout:1798 writesame 0x64dfc2c thru 0x64e6b30 data_size 0x1(0x6f05 bytes) 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.358032444000003, 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:23.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:01:23.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:01:23.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:01:23.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31980 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2804984040000003, 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095815163999999994, 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31980 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.75e-07 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968752+0000", 2024-09-22T01:01:23.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968761+0000", 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6600000000000001e-06 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968768+0000", 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2019999999999998e-06 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064177+0000", 2024-09-22T01:01:23.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095408848000000004 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064349+0000", 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017210500000000001 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064564+0000", 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000214909 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.260 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31981 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2804704170000001, 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095999240999999999, 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31981 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7199999999999999e-07 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968780+0000", 2024-09-22T01:01:23.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8679999999999999e-06 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968789+0000", 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6109999999999994e-06 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968796+0000", 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8460000000000002e-06 2024-09-22T01:01:23.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064628+0000", 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095832133 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064706+0000", 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8205000000000001e-05 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064776+0000", 2024-09-22T01:01:23.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0005999999999993e-05 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31996 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2804426810000002, 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097899502999999999, 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.265 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31996 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8599999999999998e-07 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968818+0000", 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405000000000001e-05 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968827+0000", 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6220000000000004e-06 2024-09-22T01:01:23.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968834+0000", 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.08e-06 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064831+0000", 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095997153000000002 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064952+0000", 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0001203 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066704+0000", 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001752357 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31997 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2804023689999999, 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098081842000000002, 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.269 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31997 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:23.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968846+0000", 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9699999999999995e-07 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968850+0000", 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968870+0000", 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9513000000000001e-05 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968883+0000", 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2643000000000001e-05 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066744+0000", 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097861589999999998 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066823+0000", 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8861000000000003e-05 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066927+0000", 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103824 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31998 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2803471579999997, 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098374735000000005, 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31998 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968901+0000", 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5499999999999996e-07 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968907+0000", 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.665e-06 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968923+0000", 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6501999999999999e-05 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968936+0000", 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2371000000000001e-05 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066994+0000", 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098058800000000002 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067119+0000", 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012444499999999999 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067275+0000", 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000155997 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31999 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2802946579999999, 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098498996000000005, 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31999 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968954+0000", 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9699999999999994e-07 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968959+0000", 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968976+0000", 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7091e-05 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968988+0000", 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2182e-05 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067298+0000", 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098310028999999993 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067365+0000", 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6822000000000005e-05 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067452+0000", 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6794999999999998e-05 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32000 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2802435689999996, 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099074217000000006, 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.282 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32000 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969005+0000", 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8599999999999997e-07 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969010+0000", 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.096e-06 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969026+0000", 2024-09-22T01:01:23.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5942000000000001e-05 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969039+0000", 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.31e-05 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067505+0000", 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098465892999999999 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067618+0000", 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113439 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068078+0000", 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00045986099999999999 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32001 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 2785280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2801927260000001, 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099395326000000006, 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32001 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6799999999999999e-07 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969061+0000", 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2569999999999998e-06 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969076+0000", 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5770999999999999e-05 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969089+0000", 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.237e-05 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068150+0000", 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099061069000000002 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068284+0000", 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013394000000000001 2024-09-22T01:01:23.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068450+0000", 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016605100000000001 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.290 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32002 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2801416889999997, 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100151901, 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32002 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969107+0000", 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2249999999999999e-06 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969124+0000", 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2485e-05 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969137+0000", 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2771e-05 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068511+0000", 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099373690000000001 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068641+0000", 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013009600000000001 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069258+0000", 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061671100000000004 2024-09-22T01:01:23.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32003 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2800991140000004, 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100425446, 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32003 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969149+0000", 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969152+0000", 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2569999999999999e-06 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969163+0000", 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0994e-05 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969170+0000", 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2139999999999999e-06 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069326+0000", 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10015616400000001 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069437+0000", 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011088 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069574+0000", 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000136332 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32004 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 4620288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2800680589999995, 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100783426, 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32004 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.299 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969180+0000", 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6300000000000005e-07 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969183+0000", 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.954e-06 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969191+0000", 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5280000000000003e-06 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969198+0000", 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1799999999999999e-06 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069661+0000", 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10046227000000001 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069795+0000", 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000133769 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069963+0000", 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016816199999999999 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32005 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 5111808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.280039886, 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.101548493, 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32005 2024-09-22T01:01:23.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969208+0000", 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:01:23.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969211+0000", 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969220+0000", 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4849999999999992e-06 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969227+0000", 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7649999999999996e-06 2024-09-22T01:01:23.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070006+0000", 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100779862 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070125+0000", 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118455 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070756+0000", 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000631086 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.306 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32006 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 5537792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2800117679999996, 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102299499, 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32006 2024-09-22T01:01:23.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9200000000000001e-07 2024-09-22T01:01:23.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969239+0000", 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9280000000000002e-06 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969247+0000", 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1340000000000004e-06 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969254+0000", 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070860+0000", 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.101606295 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071011+0000", 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015028799999999999 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071535+0000", 2024-09-22T01:01:23.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00052444799999999995 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32007 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6029312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.279984604, 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102848786, 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.311 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32007 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9200000000000001e-07 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969266+0000", 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9569999999999998e-06 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969275+0000", 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3590000000000008e-06 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969282+0000", 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071612+0000", 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102330377 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071691+0000", 2024-09-22T01:01:23.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949000000000006e-05 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072112+0000", 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042056899999999998 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32008 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6455296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2799482429999998, 2024-09-22T01:01:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10324055, 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32008 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969300+0000", 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8899999999999999e-07 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969303+0000", 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2629999999999999e-06 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969312+0000", 2024-09-22T01:01:23.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969319+0000", 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9879999999999998e-06 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072152+0000", 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10283331900000001 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072229+0000", 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6081999999999994e-05 2024-09-22T01:01:23.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072540+0000", 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031112400000000003 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32009 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6946816~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2799187559999998, 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10339074400000001, 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.319 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32009 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969332+0000", 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0139999999999999e-06 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969341+0000", 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.371e-06 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969347+0000", 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9140000000000002e-06 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072583+0000", 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103235179 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072644+0000", 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1014999999999998e-05 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072719+0000", 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5667999999999994e-05 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32010 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2798911969999995, 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10354661, 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32010 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969357+0000", 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7700000000000004e-07 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969360+0000", 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8169999999999999e-06 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969368+0000", 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.157e-06 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969375+0000", 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8820000000000003e-06 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072761+0000", 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103386214 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072820+0000", 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9614000000000002e-05 2024-09-22T01:01:23.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072903+0000", 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2348999999999994e-05 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32011 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.2798642359999999, 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10366739699999999, 2024-09-22T01:01:23.327 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32011 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969384+0000", 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969391+0000", 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.345e-06 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6419999999999996e-06 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969407+0000", 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.046e-06 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072940+0000", 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103533503 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072997+0000", 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6783999999999997e-05 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073050+0000", 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3508000000000003e-05 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32012 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.279831926, 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10376115800000001, 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32012 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969416+0000", 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:23.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.897e-06 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3250000000000008e-06 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969434+0000", 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0049999999999998e-06 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073074+0000", 2024-09-22T01:01:23.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103639395 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073127+0000", 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3451e-05 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073177+0000", 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9502000000000001e-05 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:23.335 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:23.337 INFO:teuthology.orchestra.run.smithi002.stdout:1799 writesame 0x6acbc3a thru 0x6ad20d8 data_size 0x1(0x649f bytes) 2024-09-22T01:01:23.337 INFO:teuthology.orchestra.run.smithi002.stdout:1800 write 0xbfbb630 thru 0xbfc8b9c (0xd56d bytes) 2024-09-22T01:01:23.337 INFO:teuthology.orchestra.run.smithi002.stdout:1801 write 0xcec7fad thru 0xced6447 (0xe49b bytes) 2024-09-22T01:01:23.337 INFO:teuthology.orchestra.run.smithi002.stdout:1802 trunc from 0xea9a868 to 0x2676d7e 2024-09-22T01:01:23.337 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:23.331+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:23.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:23.366 INFO:teuthology.orchestra.run.smithi002.stdout:1804 read 0x131de82 thru 0x132b2f2 (0xd471 bytes) 2024-09-22T01:01:23.374 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:23.397 INFO:teuthology.orchestra.run.smithi002.stdout:1805 write 0xeba2cce thru 0xebb2c7b (0xffae bytes) 2024-09-22T01:01:23.404 INFO:teuthology.orchestra.run.smithi002.stdout:1807 write 0xa50e2d7 thru 0xa50e8ee (0x618 bytes) 2024-09-22T01:01:23.407 INFO:teuthology.orchestra.run.smithi002.stdout:1808 write 0xd797c73 thru 0xd79fbbd (0x7f4b bytes) 2024-09-22T01:01:23.410 INFO:teuthology.orchestra.run.smithi002.stdout:1809 writesame 0x6114229 thru 0x6122cf3 data_size 0x1(0xeacb bytes) 2024-09-22T01:01:23.437 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:23.436+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:23.475 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:23.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:23.475 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:23.475 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:23.490 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:23.499 INFO:teuthology.orchestra.run.smithi002.stdout:1810 trunc from 0xebb2c7c to 0x7e5cc2f 2024-09-22T01:01:23.521 INFO:teuthology.orchestra.run.smithi002.stdout:1811 trunc from 0x7e5cc2f to 0x65fb056 2024-09-22T01:01:23.530 INFO:teuthology.orchestra.run.smithi002.stdout:1812 writesame 0x33e0a14 thru 0x33ee650 data_size 0x1(0xdc3d bytes) 2024-09-22T01:01:23.535 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:23.535 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:23.538 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:23.535+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:23.552 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:23.552 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:23.556 INFO:teuthology.orchestra.run.smithi002.stdout:1815 read 0x3b54ac8 thru 0x3b5fda3 (0xb2dc bytes) 2024-09-22T01:01:23.570 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:23.570 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:23.574 INFO:teuthology.orchestra.run.smithi002.stdout:1816 writesame 0xc3631b2 thru 0xc372322 data_size 0xb(0xf171 bytes) 2024-09-22T01:01:23.574 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:23.574 INFO:teuthology.orchestra.run.smithi002.stdout:1819 punch from 0xad7b5ad to 0xad7d886, (0x22d9 bytes) 2024-09-22T01:01:23.574 INFO:teuthology.orchestra.run.smithi002.stdout:1820 write 0x9c65612 thru 0x9c6acb4 (0x56a3 bytes) 2024-09-22T01:01:23.576 INFO:teuthology.orchestra.run.smithi002.stdout:1821 write 0x432298e thru 0x4331c5e (0xf2d1 bytes) 2024-09-22T01:01:23.588 INFO:teuthology.orchestra.run.smithi002.stdout:1822 trunc from 0xc372323 to 0x6d4757d 2024-09-22T01:01:23.589 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:23.589 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:23.599 INFO:teuthology.orchestra.run.smithi002.stdout:1823 read 0x38f972c thru 0x38fbdc9 (0x269e bytes) 2024-09-22T01:01:23.600 INFO:teuthology.orchestra.run.smithi002.stdout:1824 writesame 0x287f09b thru 0x2883f35 data_size 0x1(0x4e9b bytes) 2024-09-22T01:01:23.612 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:23.612 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:23.612 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:23.633 INFO:teuthology.orchestra.run.smithi002.stdout:1825 writesame 0xc361327 thru 0xc367e75 data_size 0x1(0x6b4f bytes) 2024-09-22T01:01:23.634 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:23.635 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:23.635 INFO:teuthology.orchestra.run.smithi002.stdout:1827 trunc from 0xc367e76 to 0xd0da096 2024-09-22T01:01:23.639 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:23.636+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:23.640 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:23.640 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:23.642 INFO:teuthology.orchestra.run.smithi002.stdout:1829 punch from 0x58fb3b3 to 0x5900ef0, (0x5b3d bytes) 2024-09-22T01:01:23.642 INFO:teuthology.orchestra.run.smithi002.stdout:1830 punch from 0xb33b683 to 0xb348613, (0xcf90 bytes) 2024-09-22T01:01:23.643 INFO:teuthology.orchestra.run.smithi002.stdout:1831 read 0xc8721d8 thru 0xc876294 (0x40bd bytes) 2024-09-22T01:01:23.657 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:23.657 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:23.661 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:23.661 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:23.664 INFO:teuthology.orchestra.run.smithi002.stdout:1832 trunc from 0xd0da096 to 0x4a0c2 2024-09-22T01:01:23.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:23.707 INFO:teuthology.orchestra.run.smithi002.stdout:1833 read 0x7aee thru 0xe32e (0x6841 bytes) 2024-09-22T01:01:23.708 INFO:teuthology.orchestra.run.smithi002.stdout:1834 read 0x439ec thru 0x44045 (0x65a bytes) 2024-09-22T01:01:23.722 INFO:teuthology.orchestra.run.smithi002.stdout:1835 write 0x2b04a9d thru 0x2b0d0aa (0x860e bytes) 2024-09-22T01:01:23.730 INFO:teuthology.orchestra.run.smithi002.stdout:1836 read 0x120657b thru 0x1212932 (0xc3b8 bytes) 2024-09-22T01:01:23.730 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:23.740 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:23.740+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:23.751 INFO:teuthology.orchestra.run.smithi002.stdout:1839 write 0x949dbf7 thru 0x94abaf5 (0xdeff bytes) 2024-09-22T01:01:23.769 INFO:teuthology.orchestra.run.smithi002.stdout:1840 write 0x996e479 thru 0x9971463 (0x2feb bytes) 2024-09-22T01:01:23.776 INFO:teuthology.orchestra.run.smithi002.stdout:1841 write 0x559f09f thru 0x55a69a2 (0x7904 bytes) 2024-09-22T01:01:23.782 INFO:teuthology.orchestra.run.smithi002.stdout:1842 write 0x6140b2 thru 0x61dd41 (0x9c90 bytes) 2024-09-22T01:01:23.789 INFO:teuthology.orchestra.run.smithi002.stdout:1843 trunc from 0x9971464 to 0x97f966e 2024-09-22T01:01:23.812 INFO:teuthology.orchestra.run.smithi002.stdout:1844 trunc from 0x97f966e to 0x4a6a34c 2024-09-22T01:01:23.830 INFO:teuthology.orchestra.run.smithi002.stdout:1845 read 0x1344435 thru 0x134a29e (0x5e6a bytes) 2024-09-22T01:01:23.830 INFO:teuthology.orchestra.run.smithi002.stdout:1846 clone 17 order 21 su 524288 sc 5 2024-09-22T01:01:23.840 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:23.840 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:23.840 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:23.835+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:23.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:23.941 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:23.939+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:24.036 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:24.036 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:24.042 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:24.039+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:24.053 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:24.067 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone16 2024-09-22T01:01:24.143 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:24.140+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:24.166 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:24.180 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-22T01:01:24.195 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:24.195 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:24.212 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:24.261 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:24.259+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:no valid command found; 10 closest matches: 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:0 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:1 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:2 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:abort 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:assert 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:config diff 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:config diff get 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:config get 2024-09-22T01:01:24.284 INFO:teuthology.orchestra.run.smithi119.stderr:config help [] 2024-09-22T01:01:24.285 INFO:teuthology.orchestra.run.smithi119.stderr:config set ... 2024-09-22T01:01:24.285 INFO:teuthology.orchestra.run.smithi119.stderr:admin_socket: invalid command 2024-09-22T01:01:24.287 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T01:01:24.287 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-22T01:01:24.343 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:24.343 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:24.345 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:24.343+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:24.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:24.446 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:24.444+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:24.537 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:24.537 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:24.548 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:24.543+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:24.554 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:24.628 INFO:teuthology.orchestra.run.smithi002.stdout:1847 read 0x32c3e3f thru 0x32c46c5 (0x887 bytes) 2024-09-22T01:01:24.630 INFO:teuthology.orchestra.run.smithi002.stdout:1850 punch from 0x3340495 to 0x3344e9e, (0x4a09 bytes) 2024-09-22T01:01:24.634 INFO:teuthology.orchestra.run.smithi002.stdout:1851 read 0x3e53ad5 thru 0x3e5bb2b (0x8057 bytes) 2024-09-22T01:01:24.635 INFO:teuthology.orchestra.run.smithi002.stdout:1852 writesame 0x117df1c thru 0x118b41e data_size 0x53(0xd503 bytes) 2024-09-22T01:01:24.642 INFO:teuthology.orchestra.run.smithi002.stdout:1853 punch from 0xc2740d to 0xc2915d, (0x1d50 bytes) 2024-09-22T01:01:24.649 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:24.648+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:24.662 INFO:teuthology.orchestra.run.smithi002.stdout:1856 write 0xd2ee079 thru 0xd2fde77 (0xfdff bytes) 2024-09-22T01:01:24.666 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.666 INFO:teuthology.orchestra.run.smithi002.stdout:1858 punch from 0x52e53e5 to 0x52ed9ab, (0x85c6 bytes) 2024-09-22T01:01:24.666 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.666 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.666 INFO:teuthology.orchestra.run.smithi002.stdout:1862 write 0x7153a57 thru 0x7162f3e (0xf4e8 bytes) 2024-09-22T01:01:24.672 INFO:teuthology.orchestra.run.smithi002.stdout:1863 writesame 0xcfa2504 thru 0xcfaa0d7 data_size 0x32(0x7bd4 bytes) 2024-09-22T01:01:24.677 INFO:teuthology.orchestra.run.smithi002.stdout:1864 punch from 0x73fc218 to 0x73fdf8b, (0x1d73 bytes) 2024-09-22T01:01:24.677 INFO:teuthology.orchestra.run.smithi002.stdout:1865 read 0x16695f1 thru 0x1676ea2 (0xd8b2 bytes) 2024-09-22T01:01:24.678 INFO:teuthology.orchestra.run.smithi002.stdout:1866 punch from 0xac7bc7c to 0xac8808b, (0xc40f bytes) 2024-09-22T01:01:24.691 INFO:teuthology.orchestra.run.smithi002.stdout:1867 trunc from 0xd2fde78 to 0x48f4c23 2024-09-22T01:01:24.712 INFO:teuthology.orchestra.run.smithi002.stdout:1868 punch from 0x356a20e to 0x356efcc, (0x4dbe bytes) 2024-09-22T01:01:24.730 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:24.730 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:24.738 INFO:teuthology.orchestra.run.smithi002.stdout:1869 write 0xe26ed77 thru 0xe2707ad (0x1a37 bytes) 2024-09-22T01:01:24.743 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.743 INFO:teuthology.orchestra.run.smithi002.stdout:1871 write 0xe8239b thru 0xe89af2 (0x7758 bytes) 2024-09-22T01:01:24.745 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:1873 writesame 0x669bf8f thru 0x669f4f1 data_size 0x1(0x3563 bytes) 2024-09-22T01:01:24.776 INFO:teuthology.orchestra.run.smithi002.stdout:1874 trunc from 0xe2707ae to 0x3b34447 2024-09-22T01:01:24.776 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:24.771+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:24.802 INFO:teuthology.orchestra.run.smithi002.stdout:1875 writesame 0x5c2571c thru 0x5c2df47 data_size 0x54(0x882c bytes) 2024-09-22T01:01:24.803 INFO:teuthology.orchestra.run.smithi002.stdout:1876 punch from 0x4816dcb to 0x4825892, (0xeac7 bytes) 2024-09-22T01:01:24.803 INFO:teuthology.orchestra.run.smithi002.stdout:1877 read 0x4711e99 thru 0x4718627 (0x678f bytes) 2024-09-22T01:01:24.807 INFO:teuthology.orchestra.run.smithi002.stdout:1878 trunc from 0x5c2df48 to 0x88cd237 2024-09-22T01:01:24.815 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.816 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.831 INFO:teuthology.orchestra.run.smithi002.stdout:1884 write 0xcb7a278 thru 0xcb85310 (0xb099 bytes) 2024-09-22T01:01:24.837 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.837 INFO:teuthology.orchestra.run.smithi002.stdout:1886 read 0x7a01a54 thru 0x7a05a75 (0x4022 bytes) 2024-09-22T01:01:24.837 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.837 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:24.837 INFO:teuthology.orchestra.run.smithi002.stdout:1889 punch from 0xa7a6a92 to 0xa7af5c7, (0x8b35 bytes) 2024-09-22T01:01:24.837 INFO:teuthology.orchestra.run.smithi002.stdout:1890 write 0xf0fa17 thru 0xf19343 (0x992d bytes) 2024-09-22T01:01:24.839 INFO:teuthology.orchestra.run.smithi002.stdout:1891 read 0x1586af6 thru 0x1587d67 (0x1272 bytes) 2024-09-22T01:01:24.849 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:24.847+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:24.855 INFO:teuthology.orchestra.run.smithi002.stdout:1892 trunc from 0xcb85311 to 0x3660f45 2024-09-22T01:01:24.888 INFO:teuthology.orchestra.run.smithi002.stdout:1893 trunc from 0x3660f45 to 0x4897f46 2024-09-22T01:01:24.912 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:24.912 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:24.913 INFO:teuthology.orchestra.run.smithi002.stdout:1894 trunc from 0x4897f46 to 0xe1ce333 2024-09-22T01:01:24.922 INFO:teuthology.orchestra.run.smithi002.stdout:1895 read 0xe177bcc thru 0xe18509a (0xd4cf bytes) 2024-09-22T01:01:24.922 INFO:teuthology.orchestra.run.smithi002.stdout:1896 write 0xd6ead9d thru 0xd6eb130 (0x394 bytes) 2024-09-22T01:01:24.928 INFO:teuthology.orchestra.run.smithi002.stdout:1899 punch from 0x448ce46 to 0x448fec2, (0x307c bytes) 2024-09-22T01:01:24.928 INFO:teuthology.orchestra.run.smithi002.stdout:1900 trunc from 0xe1ce333 to 0xdf76355 2024-09-22T01:01:24.929 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:24.937 INFO:teuthology.orchestra.run.smithi002.stdout:1903 read 0x6682e52 thru 0x668b61f (0x87ce bytes) 2024-09-22T01:01:24.937 INFO:teuthology.orchestra.run.smithi002.stdout:1904 write 0xa10212d thru 0xa105dd9 (0x3cad bytes) 2024-09-22T01:01:24.943 INFO:teuthology.orchestra.run.smithi002.stdout:1905 write 0xab2fce5 thru 0xab3cdd5 (0xd0f1 bytes) 2024-09-22T01:01:24.949 INFO:teuthology.orchestra.run.smithi002.stdout:1909 write 0xdd8aa87 thru 0xdd97eea (0xd464 bytes) 2024-09-22T01:01:24.950 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:24.947+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:24.951 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:01:24.954 INFO:teuthology.orchestra.run.smithi002.stdout:1910 read 0x60e39d0 thru 0x60f05ff (0xcc30 bytes) 2024-09-22T01:01:24.954 INFO:teuthology.orchestra.run.smithi002.stdout:1911 read 0x9e8debb thru 0x9e9a17a (0xc2c0 bytes) 2024-09-22T01:01:24.954 INFO:teuthology.orchestra.run.smithi002.stdout:1912 write 0x1b0634d thru 0x1b09a99 (0x374d bytes) 2024-09-22T01:01:24.961 INFO:teuthology.orchestra.run.smithi002.stdout:1913 write 0x66a340d thru 0x66af5e6 (0xc1da bytes) 2024-09-22T01:01:24.984 INFO:teuthology.orchestra.run.smithi002.stdout:1916 trunc from 0xdf76355 to 0x1d39857 2024-09-22T01:01:25.003 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:25.003 INFO:teuthology.orchestra.run.smithi002.stdout:1919 clone 18 order 24 su 8388608 sc 8 2024-09-22T01:01:25.051 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:25.048+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:25.072 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:25.072 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:25.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:25.152 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:25.148+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:25.252 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:25.247+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:25.266 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:25.354 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:25.352+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:25.434 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:25.434 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:25.448 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:25.454 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:25.451+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:25.555 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:25.551+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:25.579 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:25.579 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:25.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:25.655 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:25.651+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:25.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:25.712 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:25.712 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:25.726 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:25.772 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:25.772+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:25.816 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:25.817 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:25.835 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:25.835 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:25.848 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:25.848 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:25.856 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:25.856+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:25.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:25.872 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:25.884 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:25.884+0000 7fa3818427c0 -1 osd.5 25 log_to_monitors true 2024-09-22T01:01:25.891 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:25.892 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:25.892 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-22T01:01:25.911 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:25.911 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:25.929 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:25.929 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:25.933 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T01:01:25.934 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T01:01:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:01:25.957 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:25.956+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:25.988 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:25.988+0000 7fa37560f640 -1 osd.5 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:01:26.002 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:26.003 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:26.018 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:26.031 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone17 intact 2024-09-22T01:01:26.058 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:26.055+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:26.132 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-22T01:01:26.159 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:26.156+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:26.184 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:26.185 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:26.200 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:26.260 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:26.255+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:26.360 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:26.355+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2956 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3284992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989675164000001, 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044364010000000002, 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2956 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378157+0000", 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378159+0000", 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8679999999999999e-06 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378164+0000", 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2499999999999997e-06 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378169+0000", 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3900000000000003e-06 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422438+0000", 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044268764000000002 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422478+0000", 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.066e-05 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422521+0000", 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2741999999999998e-05 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2957 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989658443, 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044474276, 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2957 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378174+0000", 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1299999999999999e-07 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378175+0000", 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3039999999999999e-06 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378179+0000", 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.388e-06 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378188+0000", 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0569999999999999e-06 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422550+0000", 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044362523000000001 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422597+0000", 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6493999999999999e-05 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422648+0000", 2024-09-22T01:01:26.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1297e-05 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2958 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989638397, 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044568617999999997, 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2958 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:26.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378194+0000", 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.75e-07 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378196+0000", 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4959999999999999e-06 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378199+0000", 2024-09-22T01:01:26.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7969999999999998e-06 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378202+0000", 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.65e-06 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422678+0000", 2024-09-22T01:01:26.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044475564000000002 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422718+0000", 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0825000000000003e-05 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422762+0000", 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3810999999999998e-05 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2959 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989626139999999, 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044681193000000001, 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2959 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378206+0000", 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378207+0000", 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.316e-06 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378211+0000", 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378213+0000", 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.588e-06 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422794+0000", 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044580928999999998 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422840+0000", 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5231000000000001e-05 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422887+0000", 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7543999999999997e-05 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2960 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989615095000001, 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044780915999999997, 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2960 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378217+0000", 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378218+0000", 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.032e-06 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378222+0000", 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4690000000000002e-06 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378224+0000", 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5040000000000001e-06 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422915+0000", 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044691036000000003 2024-09-22T01:01:26.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422956+0000", 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0881999999999999e-05 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.422998+0000", 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1782000000000001e-05 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.391 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2961 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989604474, 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891911, 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2961 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:26.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378228+0000", 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378229+0000", 2024-09-22T01:01:26.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.071e-06 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378232+0000", 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378235+0000", 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4609999999999998e-06 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423029+0000", 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044794498000000002 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423072+0000", 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2796000000000002e-05 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423120+0000", 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7574999999999998e-05 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2962 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989593942999999, 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044991058, 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2962 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378238+0000", 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378239+0000", 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378243+0000", 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3459999999999998e-06 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378245+0000", 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6340000000000001e-06 2024-09-22T01:01:26.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423149+0000", 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044903169999999999 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423188+0000", 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8974999999999999e-05 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423229+0000", 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1655e-05 2024-09-22T01:01:26.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2963 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989583330999999, 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104404000000001, 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378249+0000", 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378250+0000", 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.001e-06 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378255+0000", 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7949999999999998e-06 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378264+0000", 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6129999999999993e-06 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423259+0000", 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044994348000000003 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423302+0000", 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3733000000000003e-05 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423353+0000", 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0689000000000002e-05 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2964 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989563043, 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045200033000000001, 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2964 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378269+0000", 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378275+0000", 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3850000000000001e-06 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378279+0000", 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378281+0000", 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5129999999999999e-06 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423382+0000", 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045100385 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423422+0000", 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0265999999999997e-05 2024-09-22T01:01:26.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423469+0000", 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7110999999999997e-05 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2965 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989540535, 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045300746000000003, 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2965 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378292+0000", 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6899999999999999e-07 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378294+0000", 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.951e-06 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378298+0000", 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1239999999999998e-06 2024-09-22T01:01:26.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378301+0000", 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.779e-06 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423497+0000", 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045196473000000001 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423544+0000", 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6622000000000002e-05 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423592+0000", 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8528e-05 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:2966 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989527137, 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045402558000000003, 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2966 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378305+0000", 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378306+0000", 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2869999999999999e-06 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378310+0000", 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.258e-06 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378312+0000", 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7810000000000001e-06 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423622+0000", 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045309464000000001 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423663+0000", 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1310999999999998e-05 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423708+0000", 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4248000000000001e-05 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3013 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989516040000002, 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045530159000000001, 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3013 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378316+0000", 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378317+0000", 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378321+0000", 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3110000000000001e-06 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378323+0000", 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5600000000000001e-06 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423737+0000", 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045414035999999998 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423781+0000", 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3279999999999999e-05 2024-09-22T01:01:26.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423846+0000", 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5634000000000002e-05 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3014 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989505382000001, 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045653157, 2024-09-22T01:01:26.420 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3014 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378327+0000", 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378715+0000", 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038801599999999998 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378719+0000", 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4039999999999997e-06 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378731+0000", 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1755e-05 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378739+0000", 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.712e-06 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423876+0000", 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045137505000000001 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423918+0000", 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2339999999999998e-05 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.423980+0000", 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1425000000000002e-05 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3015 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989078008, 2024-09-22T01:01:26.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045369596999999998, 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3015 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:26.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378754+0000", 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378755+0000", 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0100000000000003e-07 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378759+0000", 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378766+0000", 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.029e-06 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378769+0000", 2024-09-22T01:01:26.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7189999999999999e-06 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424013+0000", 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045243798000000002 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424059+0000", 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5725e-05 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424124+0000", 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4851999999999995e-05 2024-09-22T01:01:26.428 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3016 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.989056608999999, 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045478877000000001, 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3016 2024-09-22T01:01:26.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.378775+0000", 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379160+0000", 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038423800000000002 2024-09-22T01:01:26.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379164+0000", 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379173+0000", 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.439e-06 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379178+0000", 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3909999999999997e-06 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424155+0000", 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044976781 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424198+0000", 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769000000000003e-05 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424254+0000", 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6641000000000002e-05 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3017 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.98863957, 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045202985000000001, 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3017 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379193+0000", 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9400000000000005e-07 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379198+0000", 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379215+0000", 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6863000000000001e-05 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379219+0000", 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0749999999999999e-06 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424286+0000", 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045067013000000003 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424332+0000", 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5531000000000002e-05 2024-09-22T01:01:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424395+0000", 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3740999999999998e-05 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3018 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.988603817000001, 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045293472000000001, 2024-09-22T01:01:26.437 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3018 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379228+0000", 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379430+0000", 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020146699999999999 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379435+0000", 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379443+0000", 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4819999999999994e-06 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379461+0000", 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.753e-05 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424424+0000", 2024-09-22T01:01:26.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044962716999999999 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424464+0000", 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0824000000000001e-05 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424522+0000", 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7383999999999998e-05 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.441 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:3019 3.a 3:50c95750:::rbd_data.117cf538ee01.000000000000000f:3 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.988361561000001, 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045194026999999998, 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3019 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.379471+0000", 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381690+0000", 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022191849999999998 2024-09-22T01:01:26.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381696+0000", 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3860000000000004e-06 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381705+0000", 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2480000000000002e-06 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.381712+0000", 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.579e-06 2024-09-22T01:01:26.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424554+0000", 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042842327999999999 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424601+0000", 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6697999999999999e-05 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:55.424665+0000", 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3602999999999994e-05 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:8990 3.a 3:51e071dd:::rbd_data.117cc87baa96.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.186815662000001, 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048973669999999997, 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 8990 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.446 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181016+0000", 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181019+0000", 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4889999999999998e-06 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181038+0000", 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9097e-05 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181066+0000", 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7514000000000001e-05 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181078+0000", 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2626000000000001e-05 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181196+0000", 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118208 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.181388+0000", 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019203100000000001 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.188674+0000", 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072856120000000003 2024-09-22T01:01:26.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229668+0000", 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040994282 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229764+0000", 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6036e-05 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229859+0000", 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5088999999999997e-05 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:59.229990+0000", 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130686 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:37962 3.10 3:0f755ae6:::rbd_data.117cf8aab9ed.0000000000000008:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:23.416006+0000", 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.9518259900000001, 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073817524999999995, 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.451 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37962 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.416006+0000", 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.416006+0000", 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.416008+0000", 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.357e-06 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.416020+0000", 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1959e-05 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.416051+0000", 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0102000000000001e-05 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.416066+0000", 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5299999999999999e-05 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.416173+0000", 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107438 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.416304+0000", 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013079399999999999 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.430036+0000", 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013731505 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.489521+0000", 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.059485107000000002 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.489557+0000", 2024-09-22T01:01:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6359000000000002e-05 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.489691+0000", 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013430000000000001 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:23.489824+0000", 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132304 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.457 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.457 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:26.458 INFO:teuthology.orchestra.run.smithi002.stdout:1920 write 0x9c350c1 thru 0x9c3f918 (0xa858 bytes) 2024-09-22T01:01:26.458 INFO:teuthology.orchestra.run.smithi002.stdout:1925 trunc from 0x9c3f919 to 0xe501f7d 2024-09-22T01:01:26.458 INFO:teuthology.orchestra.run.smithi002.stdout:1926 write 0x93d6038 thru 0x93dd8cf (0x7898 bytes) 2024-09-22T01:01:26.458 INFO:teuthology.orchestra.run.smithi002.stdout:1927 punch from 0x70277b3 to 0x703586a, (0xe0b7 bytes) 2024-09-22T01:01:26.458 INFO:teuthology.orchestra.run.smithi002.stdout:1928 write 0x6f252e7 thru 0x6f279a0 (0x26ba bytes) 2024-09-22T01:01:26.458 INFO:teuthology.orchestra.run.smithi002.stdout:1929 trunc from 0xe501f7d to 0x30cf467 2024-09-22T01:01:26.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:26.462 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:26.460+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:26.553 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:26.563 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:26.559+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:26.566 INFO:teuthology.orchestra.run.smithi002.stdout:1932 trunc from 0x30cf467 to 0x84c46ab 2024-09-22T01:01:26.580 INFO:teuthology.orchestra.run.smithi002.stdout:1933 punch from 0x6199a91 to 0x61a3b7d, (0xa0ec bytes) 2024-09-22T01:01:26.588 INFO:teuthology.orchestra.run.smithi002.stdout:1934 trunc from 0x84c46ab to 0xb6bb15c 2024-09-22T01:01:26.603 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:26.603 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:26.603 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:26.604 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:26.604 INFO:teuthology.orchestra.run.smithi002.stdout:1935 write 0x121528d thru 0x121890e (0x3682 bytes) 2024-09-22T01:01:26.608 INFO:teuthology.orchestra.run.smithi002.stdout:1936 write 0xb0c2dec thru 0xb0c7355 (0x456a bytes) 2024-09-22T01:01:26.614 INFO:teuthology.orchestra.run.smithi002.stdout:1937 punch from 0x39f8ea6 to 0x3a05e18, (0xcf72 bytes) 2024-09-22T01:01:26.615 INFO:teuthology.orchestra.run.smithi002.stdout:1938 write 0x370c772 thru 0x3714196 (0x7a25 bytes) 2024-09-22T01:01:26.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:26.621 INFO:teuthology.orchestra.run.smithi002.stdout:1939 read 0xf70bdd thru 0xf71dd7 (0x11fb bytes) 2024-09-22T01:01:26.623 INFO:teuthology.orchestra.run.smithi002.stdout:1940 write 0x29334e1 thru 0x29360e4 (0x2c04 bytes) 2024-09-22T01:01:26.624 INFO:teuthology.orchestra.run.smithi002.stdout:1941 writesame 0x72f020a thru 0x72f4769 data_size 0x8(0x4560 bytes) 2024-09-22T01:01:26.629 INFO:teuthology.orchestra.run.smithi002.stdout:1942 punch from 0x407538 to 0x412097, (0xab5f bytes) 2024-09-22T01:01:26.629 INFO:teuthology.orchestra.run.smithi002.stdout:1943 trunc from 0xb6bb15c to 0xb700058 2024-09-22T01:01:26.645 INFO:teuthology.orchestra.run.smithi002.stdout:1945 punch from 0x8616eed to 0x86185e9, (0x16fc bytes) 2024-09-22T01:01:26.645 INFO:teuthology.orchestra.run.smithi002.stdout:1946 writesame 0x8687131 thru 0x8690675 data_size 0x35(0x9545 bytes) 2024-09-22T01:01:26.653 INFO:teuthology.orchestra.run.smithi002.stdout:1947 punch from 0x69d5c to 0x6f70f, (0x59b3 bytes) 2024-09-22T01:01:26.653 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:26.653 INFO:teuthology.orchestra.run.smithi002.stdout:1949 read 0x269f913 thru 0x26a5690 (0x5d7e bytes) 2024-09-22T01:01:26.664 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:26.664+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:26.671 INFO:teuthology.orchestra.run.smithi002.stdout:1952 trunc from 0xb700058 to 0x4cea980 2024-09-22T01:01:26.699 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:26.699 INFO:teuthology.orchestra.run.smithi002.stdout:1957 writesame 0x35bc88e thru 0x35c2064 data_size 0x1(0x57d7 bytes) 2024-09-22T01:01:26.731 INFO:teuthology.orchestra.run.smithi002.stdout:1961 trunc from 0x4cea980 to 0x88de214 2024-09-22T01:01:26.740 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:26.740 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:26.741 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:26.741 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:26.741 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:26.745 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:26.745 INFO:teuthology.orchestra.run.smithi002.stdout:1963 punch from 0x49105d3 to 0x4917a3d, (0x746a bytes) 2024-09-22T01:01:26.754 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:26.765 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:26.759+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:26.765 INFO:teuthology.orchestra.run.smithi002.stdout:1964 write 0xb387494 thru 0xb38a0b0 (0x2c1d bytes) 2024-09-22T01:01:26.771 INFO:teuthology.orchestra.run.smithi002.stdout:1965 punch from 0x7895766 to 0x7895b27, (0x3c1 bytes) 2024-09-22T01:01:26.772 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:26.772 INFO:teuthology.orchestra.run.smithi002.stdout:1967 punch from 0xa871f to 0xb2a56, (0xa337 bytes) 2024-09-22T01:01:26.801 INFO:teuthology.orchestra.run.smithi002.stdout:1968 trunc from 0xb38a0b1 to 0x3cee28 2024-09-22T01:01:26.854 INFO:teuthology.orchestra.run.smithi002.stdout:1971 trunc from 0x3cee28 to 0x1e73cf8 2024-09-22T01:01:26.866 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:26.864+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:26.868 INFO:teuthology.orchestra.run.smithi002.stdout:1973 read 0x1c087e9 thru 0x1c0baa1 (0x32b9 bytes) 2024-09-22T01:01:26.868 INFO:teuthology.orchestra.run.smithi002.stdout:1975 read 0x4575af thru 0x461d43 (0xa795 bytes) 2024-09-22T01:01:26.869 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:26.869 INFO:teuthology.orchestra.run.smithi002.stdout:1977 write 0x15337d thru 0x15faaf (0xc733 bytes) 2024-09-22T01:01:26.902 INFO:teuthology.orchestra.run.smithi002.stdout:1979 write 0x776d1b2 thru 0x77766e4 (0x9533 bytes) 2024-09-22T01:01:26.908 INFO:teuthology.orchestra.run.smithi002.stdout:1980 read 0x5173a84 thru 0x517e5cb (0xab48 bytes) 2024-09-22T01:01:26.908 INFO:teuthology.orchestra.run.smithi002.stdout:1981 read 0x112d3ba thru 0x1134472 (0x70b9 bytes) 2024-09-22T01:01:26.908 INFO:teuthology.orchestra.run.smithi002.stdout:1982 read 0x248134e thru 0x248d031 (0xbce4 bytes) 2024-09-22T01:01:26.908 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:26.918 INFO:teuthology.orchestra.run.smithi002.stdout:1984 trunc from 0x77766e5 to 0x3d12098 2024-09-22T01:01:26.933 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.040622882999998, 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:01:26.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:26.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:01:26.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:01:26.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31980 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9630888429999995, 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095815163999999994, 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31980 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.75e-07 2024-09-22T01:01:26.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968752+0000", 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968761+0000", 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6600000000000001e-06 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968768+0000", 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2019999999999998e-06 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064177+0000", 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095408848000000004 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064349+0000", 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017210500000000001 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064564+0000", 2024-09-22T01:01:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000214909 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31981 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9630608560000002, 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095999240999999999, 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.945 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31981 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:26.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7199999999999999e-07 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968780+0000", 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8679999999999999e-06 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968789+0000", 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6109999999999994e-06 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968796+0000", 2024-09-22T01:01:26.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8460000000000002e-06 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064628+0000", 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095832133 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064706+0000", 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8205000000000001e-05 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064776+0000", 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0005999999999993e-05 2024-09-22T01:01:26.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31996 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9630331200000004, 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097899502999999999, 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.949 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31996 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:26.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8599999999999998e-07 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968818+0000", 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405000000000001e-05 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968827+0000", 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6220000000000004e-06 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968834+0000", 2024-09-22T01:01:26.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.08e-06 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064831+0000", 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095997153000000002 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064952+0000", 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0001203 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066704+0000", 2024-09-22T01:01:26.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001752357 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31997 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9629928079999992, 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098081842000000002, 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31997 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968846+0000", 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9699999999999995e-07 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968850+0000", 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968870+0000", 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9513000000000001e-05 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968883+0000", 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2643000000000001e-05 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066744+0000", 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097861589999999998 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066823+0000", 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8861000000000003e-05 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066927+0000", 2024-09-22T01:01:26.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103824 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31998 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9629375969999998, 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098374735000000005, 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.957 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31998 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968901+0000", 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5499999999999996e-07 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968907+0000", 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.665e-06 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968923+0000", 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6501999999999999e-05 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968936+0000", 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2371000000000001e-05 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066994+0000", 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098058800000000002 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067119+0000", 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012444499999999999 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067275+0000", 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000155997 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31999 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9628850970000009, 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098498996000000005, 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31999 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968954+0000", 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9699999999999994e-07 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968959+0000", 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968976+0000", 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7091e-05 2024-09-22T01:01:26.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968988+0000", 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2182e-05 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067298+0000", 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098310028999999993 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067365+0000", 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6822000000000005e-05 2024-09-22T01:01:26.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067452+0000", 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6794999999999998e-05 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32000 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9628340079999997, 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099074217000000006, 2024-09-22T01:01:26.965 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32000 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969005+0000", 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8599999999999997e-07 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969010+0000", 2024-09-22T01:01:26.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.096e-06 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969026+0000", 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5942000000000001e-05 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969039+0000", 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.31e-05 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067505+0000", 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098465892999999999 2024-09-22T01:01:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067618+0000", 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113439 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068078+0000", 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00045986099999999999 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32001 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 2785280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.969 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9627831649999994, 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099395326000000006, 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32001 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6799999999999999e-07 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969061+0000", 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2569999999999998e-06 2024-09-22T01:01:26.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969076+0000", 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5770999999999999e-05 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969089+0000", 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.237e-05 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068150+0000", 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099061069000000002 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068284+0000", 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013394000000000001 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068450+0000", 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016605100000000001 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32002 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9627321280000007, 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100151901, 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32002 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969107+0000", 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2249999999999999e-06 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969124+0000", 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2485e-05 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969137+0000", 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2771e-05 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068511+0000", 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099373690000000001 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068641+0000", 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013009600000000001 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069258+0000", 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061671100000000004 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.977 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32003 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9626895530000006, 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100425446, 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32003 2024-09-22T01:01:26.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969149+0000", 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969152+0000", 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2569999999999999e-06 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969163+0000", 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0994e-05 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969170+0000", 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2139999999999999e-06 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069326+0000", 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10015616400000001 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069437+0000", 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011088 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069574+0000", 2024-09-22T01:01:26.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000136332 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32004 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 4620288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9626584979999997, 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100783426, 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32004 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969180+0000", 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6300000000000005e-07 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969183+0000", 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.954e-06 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969191+0000", 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5280000000000003e-06 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969198+0000", 2024-09-22T01:01:26.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1799999999999999e-06 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069661+0000", 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10046227000000001 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069795+0000", 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000133769 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069963+0000", 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016816199999999999 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32005 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 5111808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9626303249999992, 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.101548493, 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32005 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969208+0000", 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969211+0000", 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969220+0000", 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4849999999999992e-06 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969227+0000", 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7649999999999996e-06 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070006+0000", 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100779862 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070125+0000", 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118455 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070756+0000", 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000631086 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32006 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 5537792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9626022069999998, 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102299499, 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32006 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9200000000000001e-07 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969239+0000", 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9280000000000002e-06 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969247+0000", 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1340000000000004e-06 2024-09-22T01:01:26.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969254+0000", 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070860+0000", 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.101606295 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071011+0000", 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015028799999999999 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071535+0000", 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00052444799999999995 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32007 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6029312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9625750429999993, 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102848786, 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.994 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32007 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:26.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9200000000000001e-07 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969266+0000", 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9569999999999998e-06 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969275+0000", 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3590000000000008e-06 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969282+0000", 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071612+0000", 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102330377 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071691+0000", 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949000000000006e-05 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072112+0000", 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042056899999999998 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32008 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6455296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9625386819999999, 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10324055, 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32008 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:26.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969300+0000", 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8899999999999999e-07 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969303+0000", 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2629999999999999e-06 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969312+0000", 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:01:27.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969319+0000", 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9879999999999998e-06 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072152+0000", 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10283331900000001 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072229+0000", 2024-09-22T01:01:27.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6081999999999994e-05 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072540+0000", 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031112400000000003 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32009 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6946816~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9625091949999991, 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10339074400000001, 2024-09-22T01:01:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32009 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:27.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969332+0000", 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0139999999999999e-06 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969341+0000", 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.371e-06 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969347+0000", 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9140000000000002e-06 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072583+0000", 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103235179 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072644+0000", 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1014999999999998e-05 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072719+0000", 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5667999999999994e-05 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32010 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:27.006 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9624816359999997, 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10354661, 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32010 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:27.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969357+0000", 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7700000000000004e-07 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969360+0000", 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8169999999999999e-06 2024-09-22T01:01:27.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969368+0000", 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.157e-06 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969375+0000", 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8820000000000003e-06 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072761+0000", 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103386214 2024-09-22T01:01:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072820+0000", 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9614000000000002e-05 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072903+0000", 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2348999999999994e-05 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.010 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32011 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.962454675, 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10366739699999999, 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32011 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:27.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969384+0000", 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969391+0000", 2024-09-22T01:01:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.345e-06 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6419999999999996e-06 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969407+0000", 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.046e-06 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072940+0000", 2024-09-22T01:01:27.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103533503 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072997+0000", 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6783999999999997e-05 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073050+0000", 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3508000000000003e-05 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32012 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9624223650000001, 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10376115800000001, 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32012 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:27.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969416+0000", 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:01:27.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.897e-06 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3250000000000008e-06 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969434+0000", 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0049999999999998e-06 2024-09-22T01:01:27.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073074+0000", 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103639395 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073127+0000", 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3451e-05 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073177+0000", 2024-09-22T01:01:27.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9502000000000001e-05 2024-09-22T01:01:27.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:27.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:27.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:27.019 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:27.020 INFO:teuthology.orchestra.run.smithi002.stdout:1985 write 0x7129a27 thru 0x713001c (0x65f6 bytes) 2024-09-22T01:01:27.020 INFO:teuthology.orchestra.run.smithi002.stdout:1987 write 0x183a43a thru 0x184723f (0xce06 bytes) 2024-09-22T01:01:27.021 INFO:teuthology.orchestra.run.smithi002.stdout:1988 writesame 0xe14cc27 thru 0xe15cafc data_size 0xa6(0xfed6 bytes) 2024-09-22T01:01:27.021 INFO:teuthology.orchestra.run.smithi002.stdout:1989 write 0xa363692 thru 0xa36ef5f (0xb8ce bytes) 2024-09-22T01:01:27.021 INFO:teuthology.orchestra.run.smithi002.stdout:1990 write 0x9ca87d9 thru 0x9cb4e7b (0xc6a3 bytes) 2024-09-22T01:01:27.021 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:27.015+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:27.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:27.026 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:27.026 INFO:teuthology.orchestra.run.smithi002.stdout:1992 clone 19 order 19 su 65536 sc 9 2024-09-22T01:01:27.121 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:27.120+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:27.158 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:27.233 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:27.233 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:27.233 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:27.234 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:27.235 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:27.231+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:27.335 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:27.331+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:27.407 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:27.407 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:27.407 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:27.407 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:27.407 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:27.422 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:27.465 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:27.463+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:27.537 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:27.536+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:27.588 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32450 3.1b 3:dca5d1c0:::rbd_data.117cdc8a00a7.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.749136129, 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028862692999999998, 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32450 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837159+0000", 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837171+0000", 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2132e-05 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837199+0000", 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7647000000000001e-05 2024-09-22T01:01:27.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837211+0000", 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2231e-05 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837290+0000", 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8617999999999998e-05 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837416+0000", 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000126406 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.840665+0000", 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032488389999999999 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865675+0000", 2024-09-22T01:01:27.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025009834000000002 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865729+0000", 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4348999999999998e-05 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865890+0000", 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000161034 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.866019+0000", 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012889600000000001 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32568 3.1b 3:dca5d1c0:::rbd_data.117cdc8a00a7.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.5124634229999998, 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045259445000000002, 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32568 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073832+0000", 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.796e-06 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073844+0000", 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2276000000000001e-05 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073875+0000", 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1127999999999997e-05 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073894+0000", 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8352e-05 2024-09-22T01:01:27.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.074008+0000", 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114034 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.074144+0000", 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013626600000000001 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:27.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.081862+0000", 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077184369999999999 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118757+0000", 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036894588999999998 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118852+0000", 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4911e-05 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118961+0000", 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109331 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.119088+0000", 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012732499999999999 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32757 3.1 3:822836a6:::rbd_data.117cdc8a00a7.000000000000002a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.0193259440000002, 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040855028000000002, 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32757 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566969+0000", 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566980+0000", 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1742999999999999e-05 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567011+0000", 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0555000000000001e-05 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567025+0000", 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4046000000000001e-05 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567177+0000", 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015239100000000001 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567424+0000", 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024646200000000003 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.573230+0000", 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058063960000000001 2024-09-22T01:01:27.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607542+0000", 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034311832 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607651+0000", 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108631 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607746+0000", 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4850000000000002e-05 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607822+0000", 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5943000000000001e-05 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36334 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:27.607 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8302249210000001, 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.02825625, 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36334 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4000000000000002e-07 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756071+0000", 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9459999999999998e-06 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756080+0000", 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6430000000000001e-06 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756086+0000", 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2489999999999998e-06 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.783797+0000", 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027711369999999999 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784023+0000", 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022560200000000001 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784324+0000", 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000301 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.612 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36335 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.612 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:27.612 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8301981649999997, 2024-09-22T01:01:27.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028810327, 2024-09-22T01:01:27.612 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36335 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:27.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:27.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756095+0000", 2024-09-22T01:01:27.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:01:27.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756098+0000", 2024-09-22T01:01:27.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8569999999999999e-06 2024-09-22T01:01:27.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756106+0000", 2024-09-22T01:01:27.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3070000000000003e-06 2024-09-22T01:01:27.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756112+0000", 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784404+0000", 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028292048 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784621+0000", 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216865 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784905+0000", 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028360199999999999 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36336 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8301717970000002, 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029359435, 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36336 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756124+0000", 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9009999999999998e-06 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756132+0000", 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.259e-06 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756139+0000", 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:01:27.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784985+0000", 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028846663000000002 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785202+0000", 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216648 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785480+0000", 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027814199999999999 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36338 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.830145355, 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029907710000000001, 2024-09-22T01:01:27.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36338 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756148+0000", 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756150+0000", 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9399999999999998e-06 2024-09-22T01:01:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756159+0000", 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3019999999999995e-06 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756165+0000", 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1879999999999999e-06 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785559+0000", 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029393586999999999 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785773+0000", 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021459499999999999 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.786055+0000", 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028168900000000001 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36509 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8169801660000005, 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025912054, 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36509 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769313+0000", 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0759999999999999e-06 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769331+0000", 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.753e-05 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769350+0000", 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9585e-05 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769365+0000", 2024-09-22T01:01:27.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4926000000000001e-05 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.794791+0000", 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025425136000000001 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.794998+0000", 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000207016 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795224+0000", 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022678499999999999 2024-09-22T01:01:27.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36510 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8169075000000001, 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026348881000000001, 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36510 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769386+0000", 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9999999999999995e-07 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769393+0000", 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.153e-06 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769411+0000", 2024-09-22T01:01:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8139e-05 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769425+0000", 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3579e-05 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795301+0000", 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025876399000000001 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795508+0000", 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020672600000000001 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795734+0000", 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225885 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36511 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.636 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8168482499999996, 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026956201999999999, 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36511 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:27.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769445+0000", 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0300000000000001e-06 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769451+0000", 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0270000000000001e-06 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769470+0000", 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769483+0000", 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3271999999999999e-05 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795809+0000", 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026326142 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796017+0000", 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020782600000000001 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796400+0000", 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038322600000000002 2024-09-22T01:01:27.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36512 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8167904459999997, 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027282219, 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36512 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769503+0000", 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769509+0000", 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999998e-06 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769527+0000", 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8516e-05 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769541+0000", 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3586e-05 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796486+0000", 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026944941 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796622+0000", 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013610200000000001 2024-09-22T01:01:27.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796784+0000", 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000162468 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36513 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 638976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8167335260000002, 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027552297999999999, 2024-09-22T01:01:27.645 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36513 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769560+0000", 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.06e-06 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769565+0000", 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769583+0000", 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8122999999999999e-05 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769597+0000", 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3482000000000001e-05 2024-09-22T01:01:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796834+0000", 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027237849000000001 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796956+0000", 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012197300000000001 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797111+0000", 2024-09-22T01:01:27.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015474799999999999 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36514 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8166782789999996, 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027850182000000001, 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36514 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769615+0000", 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769621+0000", 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6670000000000001e-06 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769639+0000", 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8389e-05 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769652+0000", 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2839e-05 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797204+0000", 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027551902 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797331+0000", 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012647 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797464+0000", 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133845 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36515 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.654 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8166205020000001, 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028090786999999999, 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36515 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769673+0000", 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0100000000000001e-06 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769679+0000", 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8429999999999998e-06 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769698+0000", 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8759000000000001e-05 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769711+0000", 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3699e-05 2024-09-22T01:01:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797510+0000", 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027798442999999999 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797628+0000", 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118268 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797763+0000", 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013476499999999999 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36971 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 573440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8155500179999997, 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025899893, 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36971 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:27.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770744+0000", 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1030000000000001e-06 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770758+0000", 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4866e-05 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770777+0000", 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8782000000000001e-05 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770790+0000", 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3190000000000001e-05 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.791603+0000", 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020812761999999999 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.791710+0000", 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010676200000000001 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796642+0000", 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004932428 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.663 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37889 3.1f 3:fada49d2:::rbd_data.117cd9a1164e.0000000000000026:12 [sparse-read 557056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.138183+0000", 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4481093950000004, 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025829491, 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37889 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138183+0000", 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138183+0000", 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138184+0000", 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4900000000000005e-07 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138193+0000", 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8109999999999992e-06 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138207+0000", 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4829e-05 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138216+0000", 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4619999999999996e-06 2024-09-22T01:01:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.163608+0000", 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025392385999999999 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.163762+0000", 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015337700000000001 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164013+0000", 2024-09-22T01:01:27.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025087699999999999 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37890 3.1f 3:fada49d2:::rbd_data.117cd9a1164e.0000000000000026:12 [sparse-read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.448062663, 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026289204, 2024-09-22T01:01:27.668 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37890 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138231+0000", 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.92e-07 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138236+0000", 2024-09-22T01:01:27.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.057e-06 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138249+0000", 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3794e-05 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138258+0000", 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5429999999999994e-06 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164056+0000", 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025798232000000001 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164220+0000", 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016423499999999999 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164519+0000", 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029855099999999998 2024-09-22T01:01:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37920 3.1b 3:d9ee1e9e:::rbd_data.117cf8aab9ed.0000000000000005:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.3484550390000001, 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035889233999999999, 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37920 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237839+0000", 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.013e-06 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237850+0000", 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0473e-05 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237880+0000", 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0468e-05 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237902+0000", 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1569e-05 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.238081+0000", 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017904800000000001 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.238294+0000", 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021303500000000001 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.244279+0000", 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005984545 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273477+0000", 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029198275999999999 2024-09-22T01:01:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273560+0000", 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665000000000006e-05 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273661+0000", 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100839 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273727+0000", 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5302999999999995e-05 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38844 3.12 3:49c96a36:::rbd_data.117c55693b3.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.105514675, 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026176943000000001, 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38844 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480781+0000", 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.557e-06 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484901+0000", 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041200519999999999 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484932+0000", 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0261999999999999e-05 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484947+0000", 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4893e-05 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485088+0000", 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000141936 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485329+0000", 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024086600000000001 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485651+0000", 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032141 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.500650+0000", 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014999111000000001 2024-09-22T01:01:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506603+0000", 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059526689999999998 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506789+0000", 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186725 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506955+0000", 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016546200000000001 2024-09-22T01:01:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38846 3.1 3:823bb6bd:::rbd_data.117c55693b3.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0992774009999999, 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049688678999999999, 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38846 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487020+0000", 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.023e-06 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497105+0000", 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010085393 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497147+0000", 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1866999999999997e-05 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497210+0000", 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2324999999999997e-05 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497457+0000", 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024715099999999998 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497726+0000", 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026944000000000002 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.498317+0000", 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059106199999999995 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.516775+0000", 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018457952 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536527+0000", 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019751484 2024-09-22T01:01:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536622+0000", 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5529e-05 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536704+0000", 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1452999999999994e-05 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:27.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:27.691 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:27.692 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:27.692 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:27.687+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:27.792 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:27.787+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:27.879 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:27.879 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:27.879 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:27.879 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:27.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:27.894 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:27.892+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:27.934 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:27.980 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:01:28.025 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:28.023+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:28.064 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:28.064 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:28.064 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:28.064 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:28.064 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:28.081 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:28.098 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:28.098 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:28.098 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:28.095+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:28.120 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:28.120 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:28.137 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:28.154 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:28.175 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:28.175 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:28.195 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:28.192+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:28.196 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:28.196 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-22T01:01:28.216 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:28.238 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:28.242 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:28.242 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:28.259 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:28.301 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:28.299+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:28.397 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:28.391+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:28.416 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:28.416 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:28.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:28.517 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:28.515+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:28.598 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:28.595+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:28.601 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:28.602 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:28.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:28.699 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:28.696+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:28.709 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:01:28.789 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:28.789 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:28.800 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:28.795+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:28.803 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:28.900 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:28.900+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:28.979 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:28.979 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:28.983 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone18 from 0xe15cafd (overlap 0x3cee28) to 0x185f43 2024-09-22T01:01:28.998 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:29.001 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:28.995+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:29.101 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:29.099+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:29.127 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:29.127 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:29.139 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-22T01:01:29.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:29.202 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:29.200+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:29.263 INFO:teuthology.orchestra.run.smithi002.stdout:1993 trunc from 0xe15cafd to 0x6f4f5f0 2024-09-22T01:01:29.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:29.308 INFO:teuthology.orchestra.run.smithi002.stdout:1994 punch from 0x188ceeb to 0x1894721, (0x7836 bytes) 2024-09-22T01:01:29.308 INFO:teuthology.orchestra.run.smithi002.stdout:1995 read 0x6a56122 thru 0x6a5e26a (0x8149 bytes) 2024-09-22T01:01:29.308 INFO:teuthology.orchestra.run.smithi002.stdout:1996 write 0x3222ca5 thru 0x322694b (0x3ca7 bytes) 2024-09-22T01:01:29.317 INFO:teuthology.orchestra.run.smithi002.stdout:1998 read 0x3874900 thru 0x387709d (0x279e bytes) 2024-09-22T01:01:29.317 INFO:teuthology.orchestra.run.smithi002.stdout:1999 write 0x3c24d5f thru 0x3c34238 (0xf4da bytes) 2024-09-22T01:01:29.321 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:29.321 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:29.325 INFO:teuthology.orchestra.run.smithi002.stdout:2000 punch from 0x3f6a12b to 0x3f753b2, (0xb287 bytes) 2024-09-22T01:01:29.332 INFO:teuthology.orchestra.run.smithi002.stdout:2001 trunc from 0x6f4f5f0 to 0x339ec35 2024-09-22T01:01:29.332 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:29.332+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:29.342 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:29.375 INFO:teuthology.orchestra.run.smithi002.stdout:2003 punch from 0x403772 to 0x4076c1, (0x3f4f bytes) 2024-09-22T01:01:29.396 INFO:teuthology.orchestra.run.smithi002.stdout:2004 trunc from 0x339ec35 to 0xe39583b 2024-09-22T01:01:29.403 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:29.399+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:29.404 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:29.404 INFO:teuthology.orchestra.run.smithi002.stdout:2006 read 0x6734dd5 thru 0x673ef1a (0xa146 bytes) 2024-09-22T01:01:29.404 INFO:teuthology.orchestra.run.smithi002.stdout:2009 write 0x103ada4 thru 0x1046d09 (0xbf66 bytes) 2024-09-22T01:01:29.410 INFO:teuthology.orchestra.run.smithi002.stdout:2010 read 0x8a2fed9 thru 0x8a37e8a (0x7fb2 bytes) 2024-09-22T01:01:29.410 INFO:teuthology.orchestra.run.smithi002.stdout:2012 punch from 0xe2a8588 to 0xe2b6bbd, (0xe635 bytes) 2024-09-22T01:01:29.440 INFO:teuthology.orchestra.run.smithi002.stdout:2013 trunc from 0xe39583b to 0x51a686 2024-09-22T01:01:29.443 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:29.443 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:29.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T01:01:29.480 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:29.481 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:2014 trunc from 0x51a686 to 0xd68361 2024-09-22T01:01:29.491 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:29.496 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:29.520 INFO:teuthology.orchestra.run.smithi002.stdout:2016 write 0xc383718 thru 0xc393120 (0xfa09 bytes) 2024-09-22T01:01:29.525 INFO:teuthology.orchestra.run.smithi002.stdout:2017 punch from 0x60528af to 0x60624bd, (0xfc0e bytes) 2024-09-22T01:01:29.525 INFO:teuthology.orchestra.run.smithi002.stdout:2018 read 0x6d2a033 thru 0x6d2facc (0x5a9a bytes) 2024-09-22T01:01:29.525 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:29.525 INFO:teuthology.orchestra.run.smithi002.stdout:2020 writesame 0x897e891 thru 0x8988420 data_size 0x83(0x9b90 bytes) 2024-09-22T01:01:29.529 INFO:teuthology.orchestra.run.smithi002.stdout:2022 clone 20 order 18 su 8192 sc 5 2024-09-22T01:01:29.544 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:29.544+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:29.605 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:29.603+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:29.663 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:29.663 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:29.676 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:29.706 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:29.703+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:29.797 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:29.797 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:29.807 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:29.804+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:29.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:29.908 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:29.908+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:29.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T01:01:30.009 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:30.003+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:30.030 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T01:01:30.110 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:30.108+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:30.154 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:30.154 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:30.168 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:30.211 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:30.207+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:30.259 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:30.312 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:30.312+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:30.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:30.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:30.410 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:30.413 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:30.407+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:30.426 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:30.427 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:30.443 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:30.443 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:30.459 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:30.459 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:30.478 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:30.478 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:30.499 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:30.499 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:30.513 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:30.513 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:30.513 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:30.511+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:30.519 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:30.519 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:30.527 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:30.540 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:30.540 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:30.614 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:30.611+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:30.649 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:30.650 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:30.664 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:30.714 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:30.712+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:30.735 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.895726328000002, 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.789 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:01:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:01:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.795 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31980 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.818192288000001, 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095815163999999994, 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31980 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968749+0000", 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.75e-07 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968752+0000", 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968761+0000", 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6600000000000001e-06 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968768+0000", 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2019999999999998e-06 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064177+0000", 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095408848000000004 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064349+0000", 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017210500000000001 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064564+0000", 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000214909 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31981 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.818164300999999, 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095999240999999999, 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.800 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31981 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968777+0000", 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7199999999999999e-07 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968780+0000", 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8679999999999999e-06 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968789+0000", 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6109999999999994e-06 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968796+0000", 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8460000000000002e-06 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064628+0000", 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095832133 2024-09-22T01:01:30.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064706+0000", 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8205000000000001e-05 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064776+0000", 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0005999999999993e-05 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31996 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 458752~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.818136565, 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097899502999999999, 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31996 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968805+0000", 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8599999999999998e-07 2024-09-22T01:01:30.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968818+0000", 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405000000000001e-05 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968827+0000", 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6220000000000004e-06 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968834+0000", 2024-09-22T01:01:30.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.08e-06 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064831+0000", 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.095997153000000002 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.064952+0000", 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0001203 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066704+0000", 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001752357 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31997 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.818096253, 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098081842000000002, 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31997 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968845+0000", 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968846+0000", 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9699999999999995e-07 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968850+0000", 2024-09-22T01:01:30.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968870+0000", 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9513000000000001e-05 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968883+0000", 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2643000000000001e-05 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066744+0000", 2024-09-22T01:01:30.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097861589999999998 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066823+0000", 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8861000000000003e-05 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066927+0000", 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103824 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31998 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.818041042000001, 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098374735000000005, 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31998 2024-09-22T01:01:30.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968900+0000", 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968901+0000", 2024-09-22T01:01:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5499999999999996e-07 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968907+0000", 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.665e-06 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968923+0000", 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6501999999999999e-05 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968936+0000", 2024-09-22T01:01:30.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2371000000000001e-05 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.066994+0000", 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098058800000000002 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067119+0000", 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012444499999999999 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067275+0000", 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000155997 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31999 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817988542, 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098498996000000005, 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.818 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31999 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968953+0000", 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968954+0000", 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9699999999999994e-07 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968959+0000", 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0799999999999996e-06 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968976+0000", 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7091e-05 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968988+0000", 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2182e-05 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067298+0000", 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098310028999999993 2024-09-22T01:01:30.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067365+0000", 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6822000000000005e-05 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067452+0000", 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6794999999999998e-05 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.822 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32000 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817937453000001, 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099074217000000006, 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32000 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.823 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969004+0000", 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969005+0000", 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8599999999999997e-07 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969010+0000", 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.096e-06 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969026+0000", 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5942000000000001e-05 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969039+0000", 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.31e-05 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067505+0000", 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098465892999999999 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.067618+0000", 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113439 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068078+0000", 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00045986099999999999 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32001 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 2785280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.81788661, 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099395326000000006, 2024-09-22T01:01:30.827 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32001 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969055+0000", 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6799999999999999e-07 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969061+0000", 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2569999999999998e-06 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969076+0000", 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5770999999999999e-05 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969089+0000", 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.237e-05 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068150+0000", 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099061069000000002 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068284+0000", 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013394000000000001 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068450+0000", 2024-09-22T01:01:30.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016605100000000001 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32002 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817835573, 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100151901, 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.832 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32002 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969106+0000", 2024-09-22T01:01:30.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969107+0000", 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2249999999999999e-06 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969124+0000", 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2485e-05 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969137+0000", 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2771e-05 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068511+0000", 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099373690000000001 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.068641+0000", 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013009600000000001 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069258+0000", 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061671100000000004 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.836 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32003 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817792998, 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100425446, 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32003 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969148+0000", 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969149+0000", 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969152+0000", 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2569999999999999e-06 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969163+0000", 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0994e-05 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969170+0000", 2024-09-22T01:01:30.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2139999999999999e-06 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069326+0000", 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10015616400000001 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069437+0000", 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011088 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069574+0000", 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000136332 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32004 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 4620288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817761943000001, 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100783426, 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.841 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32004 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969179+0000", 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969180+0000", 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6300000000000005e-07 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969183+0000", 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.954e-06 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969191+0000", 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5280000000000003e-06 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969198+0000", 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1799999999999999e-06 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069661+0000", 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10046227000000001 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069795+0000", 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000133769 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.069963+0000", 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016816199999999999 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.845 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32005 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 5111808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.81773377, 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.101548493, 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32005 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969207+0000", 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969208+0000", 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969211+0000", 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969220+0000", 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4849999999999992e-06 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969227+0000", 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7649999999999996e-06 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070006+0000", 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.100779862 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070125+0000", 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118455 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070756+0000", 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000631086 2024-09-22T01:01:30.849 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32006 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 5537792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817705652000001, 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102299499, 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32006 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969236+0000", 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9200000000000001e-07 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969239+0000", 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9280000000000002e-06 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969247+0000", 2024-09-22T01:01:30.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1340000000000004e-06 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969254+0000", 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.070860+0000", 2024-09-22T01:01:30.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.101606295 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071011+0000", 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015028799999999999 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071535+0000", 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00052444799999999995 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32007 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6029312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817678488, 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102848786, 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32007 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969263+0000", 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9200000000000001e-07 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969266+0000", 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9569999999999998e-06 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969275+0000", 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3590000000000008e-06 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969282+0000", 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071612+0000", 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.102330377 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.071691+0000", 2024-09-22T01:01:30.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949000000000006e-05 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072112+0000", 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042056899999999998 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32008 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6455296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:30.859 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817642126999999, 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10324055, 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32008 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:30.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969299+0000", 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969300+0000", 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8899999999999999e-07 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969303+0000", 2024-09-22T01:01:30.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2629999999999999e-06 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969312+0000", 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969319+0000", 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9879999999999998e-06 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072152+0000", 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10283331900000001 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072229+0000", 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6081999999999994e-05 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072540+0000", 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031112400000000003 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.863 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32009 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 6946816~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.81761264, 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10339074400000001, 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.864 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32009 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969329+0000", 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969332+0000", 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0139999999999999e-06 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969341+0000", 2024-09-22T01:01:30.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.371e-06 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969347+0000", 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9140000000000002e-06 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072583+0000", 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103235179 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072644+0000", 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1014999999999998e-05 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072719+0000", 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5667999999999994e-05 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32010 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817585081000001, 2024-09-22T01:01:30.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10354661, 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32010 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969356+0000", 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969357+0000", 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7700000000000004e-07 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969360+0000", 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8169999999999999e-06 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969368+0000", 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.157e-06 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969375+0000", 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8820000000000003e-06 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072761+0000", 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103386214 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072820+0000", 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9614000000000002e-05 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072903+0000", 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2348999999999994e-05 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32011 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817558119999999, 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10366739699999999, 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32011 2024-09-22T01:01:30.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969383+0000", 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969384+0000", 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969391+0000", 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.345e-06 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6419999999999996e-06 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969407+0000", 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.046e-06 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072940+0000", 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103533503 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.072997+0000", 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6783999999999997e-05 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073050+0000", 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3508000000000003e-05 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32012 3.13 3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.817525809999999, 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.10376115800000001, 2024-09-22T01:01:30.877 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32012 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969415+0000", 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969416+0000", 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:01:30.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.897e-06 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3250000000000008e-06 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969434+0000", 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0049999999999998e-06 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073074+0000", 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.103639395 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073127+0000", 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3451e-05 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.073177+0000", 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9502000000000001e-05 2024-09-22T01:01:30.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.882 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:30.882 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:30.882 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:30.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:30.883 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:30.880+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:30.984 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:30.984+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:31.013 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T01:01:31.037 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T01:01:31.060 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:31.060 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:31.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:01:31.084 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:31.079+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:31.110 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone19 from 0xc393121 (overlap 0x51a686) to 0x0 2024-09-22T01:01:31.184 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:31.179+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:31.247 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:31.247 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:31.247 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:31.247 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:31.247 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:31.255 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-22T01:01:31.261 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:01:31.268 INFO:teuthology.orchestra.run.smithi002.stdout:2023 write 0x3850844 thru 0x3851f58 (0x1715 bytes) 2024-09-22T01:01:31.281 INFO:teuthology.orchestra.run.smithi002.stdout:2024 read 0x870a53 thru 0x874f73 (0x4521 bytes) 2024-09-22T01:01:31.283 INFO:teuthology.orchestra.run.smithi002.stdout:2025 read 0x642a381 thru 0x6436a3d (0xc6bd bytes) 2024-09-22T01:01:31.283 INFO:teuthology.orchestra.run.smithi002.stdout:2027 read 0x4937077 thru 0x4943397 (0xc321 bytes) 2024-09-22T01:01:31.292 INFO:teuthology.orchestra.run.smithi002.stdout:2028 trunc from 0xc393121 to 0x6f5f1dc 2024-09-22T01:01:31.308 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:31.308+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:31.335 INFO:teuthology.orchestra.run.smithi002.stdout:2029 writesame 0xaf64302 thru 0xaf73c52 data_size 0x45(0xf951 bytes) 2024-09-22T01:01:31.335 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.335 INFO:teuthology.orchestra.run.smithi002.stdout:2031 write 0x8bfe054 thru 0x8c0a568 (0xc515 bytes) 2024-09-22T01:01:31.362 INFO:teuthology.orchestra.run.smithi002.stdout:2032 write 0xbc90d0c thru 0xbc919cb (0xcc0 bytes) 2024-09-22T01:01:31.367 INFO:teuthology.orchestra.run.smithi002.stdout:2033 write 0x940eecb thru 0x941c78c (0xd8c2 bytes) 2024-09-22T01:01:31.380 INFO:teuthology.orchestra.run.smithi002.stdout:2035 read 0x5b3d257 thru 0x5b3e1c5 (0xf6f bytes) 2024-09-22T01:01:31.386 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:31.384+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:31.389 INFO:teuthology.orchestra.run.smithi002.stdout:2038 trunc from 0xbc919cc to 0x667bd2e 2024-09-22T01:01:31.394 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:31.394 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:31.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:31.394 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:31.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.394 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:6a9bc3a5:::rbd_data.117c1b47bdd2.000000000000003d:4, version: 30'8, data_included: [70495~3233], data_size: 3233, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:6a9bc3a5:::rbd_data.117c1b47bdd2.000000000000003d:4@30'8, size: 73728, copy_subset: [(0, 73728)], clone_subset: {}, snapset: 4=[]:{4: [4]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 73728, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.736614+0000", 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.6550992959999999, 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028803729999999999, 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.736614+0000", 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.736614+0000", 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.736616+0000", 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2790000000000001e-06 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.736679+0000", 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3255000000000003e-05 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.736734+0000", 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4647e-05 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.736782+0000", 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8260999999999998e-05 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.737508+0000", 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00072632999999999997 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.737553+0000", 2024-09-22T01:01:31.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4532e-05 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.739494+0000", 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019410689999999999 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.17 53/52 [PushOp(3:efb8e70b:::rbd_data.117ce7ab0702.0000000000000004:d, version: 41'46, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:efb8e70b:::rbd_data.117ce7ab0702.0000000000000004:d@41'46, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: d=[]:{d: [d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.784004+0000", 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.6077093429999998, 2024-09-22T01:01:31.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036489370000000001, 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.784004+0000", 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.784004+0000", 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.784005+0000", 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.023e-06 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.785319+0000", 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013143989999999999 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.785353+0000", 2024-09-22T01:01:31.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3667999999999997e-05 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.785378+0000", 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4871999999999998e-05 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.785534+0000", 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015644699999999999 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.785552+0000", 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8403e-05 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.787653+0000", 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002100125 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:head, version: 42'67, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:head@42'67, size: 1078481, copy_subset: [(0, 1078481)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.402 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.799438+0000", 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5922748869999999, 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047812239999999997, 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.799438+0000", 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.799438+0000", 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.799440+0000", 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.466e-06 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801127+0000", 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016877400000000001 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801165+0000", 2024-09-22T01:01:31.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7718e-05 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801193+0000", 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8189000000000001e-05 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801339+0000", 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014596699999999999 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801359+0000", 2024-09-22T01:01:31.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.804219+0000", 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002860521 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e, version: 42'66, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e@42'66, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.810104+0000", 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.581608809, 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041628430000000003, 2024-09-22T01:01:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.810104+0000", 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.810104+0000", 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.810105+0000", 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8199999999999998e-07 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811265+0000", 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011604230000000001 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811302+0000", 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6174999999999998e-05 2024-09-22T01:01:31.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811329+0000", 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7406000000000002e-05 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811569+0000", 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024011599999999999 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811605+0000", 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5336000000000002e-05 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.814267+0000", 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026625049999999999 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e, version: 42'66, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e@42'66, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.818075+0000", 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5736382779999998, 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036794869999999999, 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.818075+0000", 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.818075+0000", 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.818076+0000", 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6470000000000001e-06 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.819235+0000", 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001158872 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.819279+0000", 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3658000000000001e-05 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.819312+0000", 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2728999999999998e-05 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.819486+0000", 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174575 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.819509+0000", 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2419e-05 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.821754+0000", 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022455869999999998 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.414 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e, version: 42'66, data_included: [2097152~28672,2138112~28672,2179072~28672,2220032~28672,2260992~28672,2301952~28672,2342912~344064], data_size: 516096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e@42'66, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3670016, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.824091+0000", 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5676222360000001, 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042729949999999999, 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824091+0000", 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824091+0000", 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824093+0000", 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.79e-06 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824687+0000", 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059397200000000001 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824733+0000", 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6870999999999999e-05 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824769+0000", 2024-09-22T01:01:31.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5982999999999998e-05 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824967+0000", 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000197448 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.825005+0000", 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8217999999999999e-05 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.828364+0000", 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033587130000000001 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:6e4efd6d:::rbd_data.117cdc8a00a7.0000000000000024:10, version: 46'77, data_included: [163840~32768], data_size: 32768, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:6e4efd6d:::rbd_data.117cdc8a00a7.0000000000000024:10@46'77, size: 196608, copy_subset: [(0, 196608)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 196608, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.838267+0000", 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.55344587, 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023202930000000002, 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.419 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.838267+0000", 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.838267+0000", 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.838268+0000", 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.035e-06 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.838331+0000", 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3146999999999997e-05 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.838368+0000", 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6266000000000001e-05 2024-09-22T01:01:31.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.838406+0000", 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8096000000000003e-05 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.838626+0000", 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022047700000000001 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.838667+0000", 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1054999999999999e-05 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.840587+0000", 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019202170000000001 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.c 53/52 [PushOp(3:35fceb55:::rbd_data.117c1b47bdd2.000000000000003e:4, version: 30'10, data_included: [65536~5603], data_size: 5603, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:35fceb55:::rbd_data.117c1b47bdd2.000000000000003e:4@30'10, size: 71139, copy_subset: [(0, 71139)], clone_subset: {}, snapset: 4=[]:{4: [4]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 71139, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.423 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.859883+0000", 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5318304120000001, 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020024520000000001, 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.859883+0000", 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.859883+0000", 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.859884+0000", 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3969999999999999e-06 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.859911+0000", 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7472000000000001e-05 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.859951+0000", 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870999999999998e-05 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.859979+0000", 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7996000000000001e-05 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.860136+0000", 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015654899999999999 2024-09-22T01:01:31.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.860156+0000", 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0377000000000002e-05 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.861885+0000", 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00172879 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.c 53/52 [PushOp(3:33db21cf:::rbd_data.117ccb654ee9.0000000000000001:7, version: 34'21, data_included: [0~524288,1258536~36357], data_size: 560645, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:33db21cf:::rbd_data.117ccb654ee9.0000000000000001:7@34'21, size: 1294893, copy_subset: [(0, 1294893)], clone_subset: {}, snapset: 7=[]:{7: [7]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1294893, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.878858+0000", 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5128549549999999, 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004942475, 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.878858+0000", 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.878858+0000", 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.878860+0000", 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.68e-06 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880689+0000", 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018293059999999999 2024-09-22T01:01:31.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880712+0000", 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3155000000000001e-05 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880729+0000", 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7142999999999998e-05 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880891+0000", 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000161705 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880915+0000", 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3903000000000001e-05 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.883801+0000", 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028855830000000002 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.431 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.432 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.d 53/52 [PushOp(3:b666f596:::rbd_data.117c6f6b4a3e.0000000000000007:e, version: 42'46, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b666f596:::rbd_data.117c6f6b4a3e.0000000000000007:e@42'46, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.432 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.879279+0000", 2024-09-22T01:01:31.433 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5124341609999998, 2024-09-22T01:01:31.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036389920000000002, 2024-09-22T01:01:31.433 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.879279+0000", 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.879279+0000", 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.879280+0000", 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880489+0000", 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00120881 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880524+0000", 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5478999999999997e-05 2024-09-22T01:01:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880546+0000", 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1875999999999999e-05 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880698+0000", 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015202999999999999 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880718+0000", 2024-09-22T01:01:31.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0225e-05 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.882918+0000", 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021995769999999999 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.d 53/52 [PushOp(3:b666f596:::rbd_data.117c6f6b4a3e.0000000000000007:e, version: 42'46, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b666f596:::rbd_data.117c6f6b4a3e.0000000000000007:e@42'46, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.886777+0000", 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5049363380000003, 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035214299999999999, 2024-09-22T01:01:31.437 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.886777+0000", 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.886777+0000", 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.886778+0000", 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.457e-06 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.888004+0000", 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001225774 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.888043+0000", 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8930000000000002e-05 2024-09-22T01:01:31.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.888075+0000", 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2144999999999998e-05 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.888244+0000", 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016951100000000001 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.888273+0000", 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8498000000000001e-05 2024-09-22T01:01:31.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.890298+0000", 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020251150000000001 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.d 53/52 [PushOp(3:b666f596:::rbd_data.117c6f6b4a3e.0000000000000007:e, version: 42'46, data_included: [2097152~20480,2129920~28672,2170880~28672,2211840~28672,2252800~28672,2293760~28672,2334720~352256], data_size: 516096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b666f596:::rbd_data.117c6f6b4a3e.0000000000000007:e@42'46, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3670016, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.892918+0000", 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.4987947950000002, 2024-09-22T01:01:31.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039331339999999996, 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.892918+0000", 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.892918+0000", 2024-09-22T01:01:31.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.892920+0000", 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3859999999999999e-06 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893516+0000", 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059648100000000005 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893558+0000", 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1544000000000001e-05 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893588+0000", 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0527000000000001e-05 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893811+0000", 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022269600000000001 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893843+0000", 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2001000000000001e-05 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.896851+0000", 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003008499 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.445 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.d 53/52 [PushOp(3:b40f02b4:::rbd_data.117c55693b3.0000000000000000:head, version: 51'67, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:b40f02b4:::rbd_data.117c55693b3.0000000000000000:head@51'67, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.918078+0000", 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.4736346810000001, 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002428936, 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.918078+0000", 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.918078+0000", 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.918080+0000", 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.919140+0000", 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010597180000000001 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.919177+0000", 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7626999999999997e-05 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.919224+0000", 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6215999999999999e-05 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.919343+0000", 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119027 2024-09-22T01:01:31.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.919356+0000", 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3348e-05 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.920507+0000", 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011511799999999999 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.d 53/52 [PushOp(3:b40f02b4:::rbd_data.117c55693b3.0000000000000000:head, version: 51'67, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b40f02b4:::rbd_data.117c55693b3.0000000000000000:head@51'67, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.923127+0000", 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.468586481, 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002275956, 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.923127+0000", 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.923127+0000", 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.923128+0000", 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.35e-06 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.924205+0000", 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010775190000000001 2024-09-22T01:01:31.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.924238+0000", 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3047999999999999e-05 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.924269+0000", 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0128999999999999e-05 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.924380+0000", 2024-09-22T01:01:31.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111561 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.924390+0000", 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.925402+0000", 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001012829 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.c 53/52 [PushOp(3:305ea727:::rbd_data.117cd9a1164e.0000000000000012:head, version: 47'73, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:305ea727:::rbd_data.117cd9a1164e.0000000000000012:head@47'73, size: 1703936, copy_subset: [(0, 1703936)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.927012+0000", 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.4647010620000001, 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033667089999999998, 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.927012+0000", 2024-09-22T01:01:31.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.927012+0000", 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.927013+0000", 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0200000000000001e-07 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.928252+0000", 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012392449999999999 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.928281+0000", 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8819999999999999e-05 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.928332+0000", 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1394000000000003e-05 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.928476+0000", 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014404700000000001 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.928495+0000", 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8762000000000001e-05 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.930379+0000", 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018836390000000001 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.d 53/52 [PushOp(3:b40f02b4:::rbd_data.117c55693b3.0000000000000000:head, version: 51'67, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b40f02b4:::rbd_data.117c55693b3.0000000000000000:head@51'67, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.929808+0000", 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.4619045719999999, 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027290090000000001, 2024-09-22T01:01:31.458 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.929808+0000", 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.929808+0000", 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.929810+0000", 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2950000000000001e-06 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.930781+0000", 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097106000000000004 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.930845+0000", 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3891000000000002e-05 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.930875+0000", 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0380999999999999e-05 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.931048+0000", 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000172931 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.931082+0000", 2024-09-22T01:01:31.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4282999999999997e-05 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.932537+0000", 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001455168 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.c 53/52 [PushOp(3:305ea727:::rbd_data.117cd9a1164e.0000000000000012:head, version: 47'73, data_included: [1048576~655360], data_size: 655360, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:305ea727:::rbd_data.117cd9a1164e.0000000000000012:head@47'73, size: 1703936, copy_subset: [(0, 1703936)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1703936, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.931333+0000", 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.4603798640000001, 2024-09-22T01:01:31.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026574789999999999, 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.931333+0000", 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.931333+0000", 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.931334+0000", 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2580000000000001e-06 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.932201+0000", 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086678799999999996 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.932238+0000", 2024-09-22T01:01:31.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6751000000000001e-05 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.932266+0000", 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7679999999999999e-05 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.932409+0000", 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014301599999999999 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.932426+0000", 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7184e-05 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.933991+0000", 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015648019999999999 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.d 53/52 [PushOp(3:b40f02b4:::rbd_data.117c55693b3.0000000000000000:head, version: 51'67, data_included: [3145728~847400], data_size: 847400, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b40f02b4:::rbd_data.117c55693b3.0000000000000000:head@51'67, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 8388608, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:31.466 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.936200+0000", 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.4555125090000001, 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020706069999999999, 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.936200+0000", 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.936200+0000", 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.936202+0000", 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2860000000000001e-06 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.936889+0000", 2024-09-22T01:01:31.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068758700000000003 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.936918+0000", 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9082999999999999e-05 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.936937+0000", 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8488999999999999e-05 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.937046+0000", 2024-09-22T01:01:31.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109073 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.937057+0000", 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.938271+0000", 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012143290000000001 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:39644 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.077465+0000", 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.31424814600000001, 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023092669999999998, 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 39644 2024-09-22T01:01:31.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077465+0000", 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077465+0000", 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077466+0000", 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4499999999999995e-07 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077470+0000", 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5809999999999998e-06 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077483+0000", 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2227e-05 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077488+0000", 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6699999999999999e-06 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077539+0000", 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0482000000000002e-05 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077600+0000", 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1032000000000001e-05 2024-09-22T01:01:31.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.077810+0000", 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020971799999999999 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.078446+0000", 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00063686799999999996 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.079705+0000", 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012582229999999999 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.079737+0000", 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2458e-05 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.079774+0000", 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7063000000000001e-05 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.476 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4476.0:39766 3.1f e56/55 3:fd748090:::rbd_data.117c162c7df4.0000000000000003:head v 56'162, mlcod=51'160)", 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.130069+0000", 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.26164419500000002, 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028944959999999999, 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.130069+0000", 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.130069+0000", 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.130071+0000", 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5969999999999999e-06 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:31.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.131009+0000", 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00093743500000000003 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.131043+0000", 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3974999999999999e-05 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.131057+0000", 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4626e-05 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.131152+0000", 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4401999999999995e-05 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.131177+0000", 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5236999999999999e-05 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.132918+0000", 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017414010000000001 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:31.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.132963+0000", 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4823000000000001e-05 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:31.481 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:31.482 INFO:teuthology.orchestra.run.smithi002.stdout:2039 write 0x13aa884 thru 0x13abb9c (0x1319 bytes) 2024-09-22T01:01:31.482 INFO:teuthology.orchestra.run.smithi002.stdout:2040 punch from 0x12b5dd9 to 0x12b60e1, (0x308 bytes) 2024-09-22T01:01:31.482 INFO:teuthology.orchestra.run.smithi002.stdout:2041 write 0xb9beff3 thru 0xb9cd91c (0xe92a bytes) 2024-09-22T01:01:31.482 INFO:teuthology.orchestra.run.smithi002.stdout:2045 trunc from 0xb9cd91d to 0x2a5058f 2024-09-22T01:01:31.483 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:31.491 INFO:teuthology.orchestra.run.smithi002.stdout:2048 trunc from 0x2a5058f to 0x4ee0e27 2024-09-22T01:01:31.492 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:31.487+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:31.510 INFO:teuthology.orchestra.run.smithi002.stdout:2050 trunc from 0x4ee0e27 to 0xad0fd37 2024-09-22T01:01:31.518 INFO:teuthology.orchestra.run.smithi002.stdout:2051 read 0x2a20d42 thru 0x2a27830 (0x6aef bytes) 2024-09-22T01:01:31.534 INFO:teuthology.orchestra.run.smithi002.stdout:2053 writesame 0xe784355 thru 0xe78b2d1 data_size 0x1(0x6f7d bytes) 2024-09-22T01:01:31.534 INFO:teuthology.orchestra.run.smithi002.stdout:2054 write 0x1ee05c6 thru 0x1ee7759 (0x7194 bytes) 2024-09-22T01:01:31.543 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.543 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.543 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.543 INFO:teuthology.orchestra.run.smithi002.stdout:2058 write 0x634064e thru 0x634c2f2 (0xbca5 bytes) 2024-09-22T01:01:31.552 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.552 INFO:teuthology.orchestra.run.smithi002.stdout:2060 read 0x2ccfc39 thru 0x2cda1a1 (0xa569 bytes) 2024-09-22T01:01:31.552 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.552 INFO:teuthology.orchestra.run.smithi002.stdout:2062 write 0xca31913 thru 0xca3d0b3 (0xb7a1 bytes) 2024-09-22T01:01:31.561 INFO:teuthology.orchestra.run.smithi002.stdout:2063 punch from 0xc31b7f to 0xc41786, (0xfc07 bytes) 2024-09-22T01:01:31.561 INFO:teuthology.orchestra.run.smithi002.stdout:2067 writesame 0x400c06e thru 0x400dc07 data_size 0x1(0x1b9a bytes) 2024-09-22T01:01:31.568 INFO:teuthology.orchestra.run.smithi002.stdout:2068 write 0xb1b53d8 thru 0xb1b970c (0x4335 bytes) 2024-09-22T01:01:31.575 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.575 INFO:teuthology.orchestra.run.smithi002.stdout:2070 writesame 0xd5ab70b thru 0xd5b6f74 data_size 0x5(0xb86a bytes) 2024-09-22T01:01:31.586 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.586 INFO:teuthology.orchestra.run.smithi002.stdout:2073 read 0xb490e74 thru 0xb49c096 (0xb223 bytes) 2024-09-22T01:01:31.587 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:31.583+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:31.593 INFO:teuthology.orchestra.run.smithi002.stdout:2076 trunc from 0xe78b2d2 to 0xa6460cd 2024-09-22T01:01:31.627 INFO:teuthology.orchestra.run.smithi002.stdout:2077 writesame 0xbb2521a thru 0xbb2f74a data_size 0x1(0xa531 bytes) 2024-09-22T01:01:31.628 INFO:teuthology.orchestra.run.smithi002.stdout:2079 write 0x1b2b809 thru 0x1b2d385 (0x1b7d bytes) 2024-09-22T01:01:31.633 INFO:teuthology.orchestra.run.smithi002.stdout:2080 punch from 0x7370a80 to 0x737d885, (0xce05 bytes) 2024-09-22T01:01:31.633 INFO:teuthology.orchestra.run.smithi002.stdout:2081 write 0x763c209 thru 0x763ffe8 (0x3de0 bytes) 2024-09-22T01:01:31.633 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:31.634 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:31.639 INFO:teuthology.orchestra.run.smithi002.stdout:2082 read 0x8317741 thru 0x831d6da (0x5f9a bytes) 2024-09-22T01:01:31.639 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.640 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.640 INFO:teuthology.orchestra.run.smithi002.stdout:2087 writesame 0x1b2ab83 thru 0x1b313df data_size 0x1(0x685d bytes) 2024-09-22T01:01:31.652 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:31.657 INFO:teuthology.orchestra.run.smithi002.stdout:2090 read 0xf05646 thru 0xf13cea (0xe6a5 bytes) 2024-09-22T01:01:31.669 INFO:teuthology.orchestra.run.smithi002.stdout:2092 write 0xe05bb62 thru 0xe06b73c (0xfbdb bytes) 2024-09-22T01:01:31.679 INFO:teuthology.orchestra.run.smithi002.stdout:2093 writesame 0x5878037 thru 0x588112a data_size 0x2(0x90f4 bytes) 2024-09-22T01:01:31.688 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:31.684+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:31.692 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.693 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.713 INFO:teuthology.orchestra.run.smithi002.stdout:2096 trunc from 0xe06b73d to 0x43be97 2024-09-22T01:01:31.789 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:31.788+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:31.797 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:31.798 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:31.820 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:31.827 INFO:teuthology.orchestra.run.smithi002.stdout:2097 write 0xd3707cc thru 0xd37143b (0xc70 bytes) 2024-09-22T01:01:31.832 INFO:teuthology.orchestra.run.smithi002.stdout:2098 writesame 0x2351b49 thru 0x23573e1 data_size 0x1(0x5899 bytes) 2024-09-22T01:01:31.849 INFO:teuthology.orchestra.run.smithi002.stdout:2099 write 0x47fc3ff thru 0x47fcf27 (0xb29 bytes) 2024-09-22T01:01:31.855 INFO:teuthology.orchestra.run.smithi002.stdout:2101 read 0x87f69be thru 0x87fc72d (0x5d70 bytes) 2024-09-22T01:01:31.855 INFO:teuthology.orchestra.run.smithi002.stdout:2104 punch from 0x6c4ff38 to 0x6c51a49, (0x1b11 bytes) 2024-09-22T01:01:31.855 INFO:teuthology.orchestra.run.smithi002.stdout:2106 write 0xd04e07d thru 0xd051ee5 (0x3e69 bytes) 2024-09-22T01:01:31.862 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:31.862 INFO:teuthology.orchestra.run.smithi002.stdout:2110 read 0xc6b94f9 thru 0xc6bc5e3 (0x30eb bytes) 2024-09-22T01:01:31.862 INFO:teuthology.orchestra.run.smithi002.stdout:2112 read 0xb0e5b61 thru 0xb0f2f47 (0xd3e7 bytes) 2024-09-22T01:01:31.862 INFO:teuthology.orchestra.run.smithi002.stdout:2113 write 0xbf0360d thru 0xbf0514e (0x1b42 bytes) 2024-09-22T01:01:31.868 INFO:teuthology.orchestra.run.smithi002.stdout:2114 read 0x53e685a thru 0x53eae6c (0x4613 bytes) 2024-09-22T01:01:31.869 INFO:teuthology.orchestra.run.smithi002.stdout:2118 punch from 0xb37e624 to 0xb37f1fc, (0xbd8 bytes) 2024-09-22T01:01:31.869 INFO:teuthology.orchestra.run.smithi002.stdout:2123 read 0x6edcac7 thru 0x6eeb01b (0xe555 bytes) 2024-09-22T01:01:31.869 INFO:teuthology.orchestra.run.smithi002.stdout:2124 read 0x3c9948c thru 0x3ca8a89 (0xf5fe bytes) 2024-09-22T01:01:31.869 INFO:teuthology.orchestra.run.smithi002.stdout:2125 clone 21 order 20 su 131072 sc 7 2024-09-22T01:01:31.889 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:31.887+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:31.964 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32450 3.1b 3:dca5d1c0:::rbd_data.117cdc8a00a7.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.124796779, 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028862692999999998, 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32450 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837159+0000", 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837171+0000", 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2132e-05 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837199+0000", 2024-09-22T01:01:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7647000000000001e-05 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837211+0000", 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2231e-05 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837290+0000", 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8617999999999998e-05 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837416+0000", 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000126406 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.840665+0000", 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032488389999999999 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865675+0000", 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025009834000000002 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865729+0000", 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4348999999999998e-05 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865890+0000", 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000161034 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.866019+0000", 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012889600000000001 2024-09-22T01:01:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32568 3.1b 3:dca5d1c0:::rbd_data.117cdc8a00a7.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.888124073, 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045259445000000002, 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32568 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073832+0000", 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.796e-06 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073844+0000", 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2276000000000001e-05 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073875+0000", 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1127999999999997e-05 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073894+0000", 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8352e-05 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.074008+0000", 2024-09-22T01:01:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114034 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.074144+0000", 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013626600000000001 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.081862+0000", 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077184369999999999 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118757+0000", 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036894588999999998 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118852+0000", 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4911e-05 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118961+0000", 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109331 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.119088+0000", 2024-09-22T01:01:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012732499999999999 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32757 3.1 3:822836a6:::rbd_data.117cdc8a00a7.000000000000002a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.394986594000001, 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040855028000000002, 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32757 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566969+0000", 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566980+0000", 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1742999999999999e-05 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567011+0000", 2024-09-22T01:01:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0555000000000001e-05 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567025+0000", 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4046000000000001e-05 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567177+0000", 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015239100000000001 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567424+0000", 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024646200000000003 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.573230+0000", 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058063960000000001 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607542+0000", 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034311832 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607651+0000", 2024-09-22T01:01:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108631 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607746+0000", 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4850000000000002e-05 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607822+0000", 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5943000000000001e-05 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36334 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.2058855709999996, 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.02825625, 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36334 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4000000000000002e-07 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756071+0000", 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9459999999999998e-06 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756080+0000", 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6430000000000001e-06 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756086+0000", 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2489999999999998e-06 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.783797+0000", 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027711369999999999 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784023+0000", 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022560200000000001 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784324+0000", 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000301 2024-09-22T01:01:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36335 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.2058588149999991, 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028810327, 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36335 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756095+0000", 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756098+0000", 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8569999999999999e-06 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756106+0000", 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3070000000000003e-06 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756112+0000", 2024-09-22T01:01:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784404+0000", 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028292048 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784621+0000", 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216865 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784905+0000", 2024-09-22T01:01:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028360199999999999 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36336 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.2058324470000006, 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029359435, 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36336 2024-09-22T01:01:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756124+0000", 2024-09-22T01:01:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9009999999999998e-06 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756132+0000", 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.259e-06 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756139+0000", 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784985+0000", 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028846663000000002 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785202+0000", 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216648 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785480+0000", 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027814199999999999 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36338 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.2058060049999995, 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029907710000000001, 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36338 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756148+0000", 2024-09-22T01:01:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756150+0000", 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9399999999999998e-06 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756159+0000", 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3019999999999995e-06 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756165+0000", 2024-09-22T01:01:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1879999999999999e-06 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785559+0000", 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029393586999999999 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785773+0000", 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021459499999999999 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.786055+0000", 2024-09-22T01:01:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028168900000000001 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36509 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1926408160000008, 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025912054, 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36509 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769313+0000", 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0759999999999999e-06 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769331+0000", 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.753e-05 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769350+0000", 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9585e-05 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769365+0000", 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4926000000000001e-05 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.794791+0000", 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025425136000000001 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.794998+0000", 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000207016 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795224+0000", 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022678499999999999 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36510 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1925681499999996, 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026348881000000001, 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36510 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769386+0000", 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9999999999999995e-07 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769393+0000", 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.153e-06 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769411+0000", 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8139e-05 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769425+0000", 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3579e-05 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795301+0000", 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025876399000000001 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795508+0000", 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020672600000000001 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795734+0000", 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225885 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36511 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1925089, 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026956201999999999, 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36511 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769445+0000", 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0300000000000001e-06 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769451+0000", 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0270000000000001e-06 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769470+0000", 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769483+0000", 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3271999999999999e-05 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795809+0000", 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026326142 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796017+0000", 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020782600000000001 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796400+0000", 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038322600000000002 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36512 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1924510959999992, 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027282219, 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36512 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769503+0000", 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769509+0000", 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999998e-06 2024-09-22T01:01:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769527+0000", 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8516e-05 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769541+0000", 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3586e-05 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796486+0000", 2024-09-22T01:01:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026944941 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796622+0000", 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013610200000000001 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796784+0000", 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000162468 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36513 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 638976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1923941760000005, 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027552297999999999, 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36513 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769560+0000", 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.06e-06 2024-09-22T01:01:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769565+0000", 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769583+0000", 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8122999999999999e-05 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769597+0000", 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3482000000000001e-05 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796834+0000", 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027237849000000001 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796956+0000", 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012197300000000001 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797111+0000", 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015474799999999999 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36514 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1923389289999999, 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027850182000000001, 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36514 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769615+0000", 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769621+0000", 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6670000000000001e-06 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769639+0000", 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8389e-05 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769652+0000", 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2839e-05 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797204+0000", 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027551902 2024-09-22T01:01:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797331+0000", 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012647 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797464+0000", 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133845 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36515 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1922811519999996, 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028090786999999999, 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36515 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769673+0000", 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0100000000000001e-06 2024-09-22T01:01:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769679+0000", 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8429999999999998e-06 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769698+0000", 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8759000000000001e-05 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769711+0000", 2024-09-22T01:01:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3699e-05 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797510+0000", 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027798442999999999 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797628+0000", 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118268 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797763+0000", 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013476499999999999 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36971 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 573440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1912106680000001, 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025899893, 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36971 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770744+0000", 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1030000000000001e-06 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770758+0000", 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4866e-05 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770777+0000", 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8782000000000001e-05 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770790+0000", 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3190000000000001e-05 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.791603+0000", 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020812761999999999 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.791710+0000", 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010676200000000001 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796642+0000", 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004932428 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37889 3.1f 3:fada49d2:::rbd_data.117cd9a1164e.0000000000000026:12 [sparse-read 557056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.138183+0000", 2024-09-22T01:01:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.8237700449999998, 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025829491, 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37889 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138183+0000", 2024-09-22T01:01:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138183+0000", 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138184+0000", 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4900000000000005e-07 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138193+0000", 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8109999999999992e-06 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138207+0000", 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4829e-05 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138216+0000", 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4619999999999996e-06 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.163608+0000", 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025392385999999999 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.163762+0000", 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015337700000000001 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164013+0000", 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025087699999999999 2024-09-22T01:01:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37890 3.1f 3:fada49d2:::rbd_data.117cd9a1164e.0000000000000026:12 [sparse-read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.8237233130000003, 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026289204, 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37890 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138231+0000", 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.92e-07 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138236+0000", 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.057e-06 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138249+0000", 2024-09-22T01:01:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3794e-05 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138258+0000", 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5429999999999994e-06 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164056+0000", 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025798232000000001 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164220+0000", 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016423499999999999 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164519+0000", 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029855099999999998 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37920 3.1b 3:d9ee1e9e:::rbd_data.117cf8aab9ed.0000000000000005:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.7241156889999996, 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035889233999999999, 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37920 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237839+0000", 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.013e-06 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237850+0000", 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0473e-05 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237880+0000", 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0468e-05 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237902+0000", 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1569e-05 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.238081+0000", 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017904800000000001 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.238294+0000", 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021303500000000001 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.244279+0000", 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005984545 2024-09-22T01:01:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273477+0000", 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029198275999999999 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273560+0000", 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665000000000006e-05 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273661+0000", 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100839 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273727+0000", 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5302999999999995e-05 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38844 3.12 3:49c96a36:::rbd_data.117c55693b3.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.4811753249999997, 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026176943000000001, 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38844 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480781+0000", 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.557e-06 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484901+0000", 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041200519999999999 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484932+0000", 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0261999999999999e-05 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484947+0000", 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4893e-05 2024-09-22T01:01:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485088+0000", 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000141936 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485329+0000", 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024086600000000001 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485651+0000", 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032141 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.500650+0000", 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014999111000000001 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506603+0000", 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059526689999999998 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506789+0000", 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186725 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506955+0000", 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016546200000000001 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38846 3.1 3:823bb6bd:::rbd_data.117c55693b3.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.4749380509999996, 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049688678999999999, 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38846 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487020+0000", 2024-09-22T01:01:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.023e-06 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497105+0000", 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010085393 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497147+0000", 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1866999999999997e-05 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497210+0000", 2024-09-22T01:01:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2324999999999997e-05 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497457+0000", 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024715099999999998 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497726+0000", 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026944000000000002 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.498317+0000", 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059106199999999995 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.516775+0000", 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018457952 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536527+0000", 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019751484 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536622+0000", 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5529e-05 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536704+0000", 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1452999999999994e-05 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.063 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:32.065 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:32.065 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T01:01:32.065 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:01:32.066 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T01:01:32.067 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:32.064+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:32.165 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:32.164+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone20 from 0xd37143c (overlap 0x43be97) to 0x0 2024-09-22T01:01:32.250 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:32.250 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:32.266 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:32.263+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:32.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:32.368 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:32.364+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:32.439 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:32.439 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:32.454 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:32.497 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:32.495+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:32.559 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.504013407999999, 2024-09-22T01:01:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503994548999998, 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503976364000003, 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503958576000002, 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503940018000002, 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503921966, 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503904617000003, 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503882664000002, 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503865128999998, 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503841342999998, 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503823480999998, 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503796244, 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503777538999998, 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503754041999997, 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503730073, 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.503710857999998, 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.502706308, 2024-09-22T01:01:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.500253119999996, 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.500231040000003, 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.500200354999997, 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:32.670 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:32.670 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:32.579+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:32.671 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:32.672 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-22T01:01:32.672 INFO:teuthology.orchestra.run.smithi002.stdout:2126 read 0x8c012fb thru 0x8c0d690 (0xc396 bytes) 2024-09-22T01:01:32.672 INFO:teuthology.orchestra.run.smithi002.stdout:2127 read 0x2da43bb thru 0x2dad5ba (0x9200 bytes) 2024-09-22T01:01:32.672 INFO:teuthology.orchestra.run.smithi002.stdout:2130 read 0x47e80b7 thru 0x47edbad (0x5af7 bytes) 2024-09-22T01:01:32.672 INFO:teuthology.orchestra.run.smithi002.stdout:2132 trunc from 0xd37143c to 0x50489cf 2024-09-22T01:01:32.683 INFO:teuthology.orchestra.run.smithi002.stdout:2133 writesame 0x567eeb2 thru 0x5684842 data_size 0x1(0x5991 bytes) 2024-09-22T01:01:32.683 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:32.679+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:32.694 INFO:teuthology.orchestra.run.smithi002.stdout:2134 write 0x779b557 thru 0x77a07c3 (0x526d bytes) 2024-09-22T01:01:32.704 INFO:teuthology.orchestra.run.smithi002.stdout:2135 read 0x27230c2 thru 0x2726f72 (0x3eb1 bytes) 2024-09-22T01:01:32.704 INFO:teuthology.orchestra.run.smithi002.stdout:2136 punch from 0x4c66fac to 0x4c72fb6, (0xc00a bytes) 2024-09-22T01:01:32.704 INFO:teuthology.orchestra.run.smithi002.stdout:2137 punch from 0xe041ff to 0xe12989, (0xe78a bytes) 2024-09-22T01:01:32.704 INFO:teuthology.orchestra.run.smithi002.stdout:2138 read 0x71ae503 thru 0x71b5c74 (0x7772 bytes) 2024-09-22T01:01:32.704 INFO:teuthology.orchestra.run.smithi002.stdout:2139 write 0x1b023c1 thru 0x1b085c8 (0x6208 bytes) 2024-09-22T01:01:32.710 INFO:teuthology.orchestra.run.smithi002.stdout:2140 clone 22 order 23 su 2097152 sc 8 2024-09-22T01:01:32.715 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:32.716 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:32.732 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:32.733 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:32.749 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:32.749 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:32.768 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:32.768 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:32.774 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:32.772+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:32.788 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:32.789 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:32.803 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:32.803 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:32.809 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:32.809 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:32.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:32.831 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:32.831 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:32.852 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:32.875 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:32.872+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:32.955 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:32.955 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:32.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:32.976 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:32.976+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T01:01:33.046 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-22T01:01:33.093 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:33.091+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:33.096 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:33.096 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:33.113 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:33.172 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone21 from 0x77a07c4 (overlap 0x50489cf) to 0x0 2024-09-22T01:01:33.178 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:33.176+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:33.279 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:33.276+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:33.302 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:33.302 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:33.325 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:33.413 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:33.411+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:33.442 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-22T01:01:33.444 INFO:teuthology.orchestra.run.smithi002.stdout:2141 read 0x38932fa thru 0x389d48d (0xa194 bytes) 2024-09-22T01:01:33.459 INFO:teuthology.orchestra.run.smithi002.stdout:2142 write 0x7ccbcea thru 0x7cd2ea4 (0x71bb bytes) 2024-09-22T01:01:33.467 INFO:teuthology.orchestra.run.smithi002.stdout:2143 read 0x5926115 thru 0x5930feb (0xaed7 bytes) 2024-09-22T01:01:33.467 INFO:teuthology.orchestra.run.smithi002.stdout:2144 punch from 0x60cb1ee to 0x60ccb65, (0x1977 bytes) 2024-09-22T01:01:33.467 INFO:teuthology.orchestra.run.smithi002.stdout:2145 read 0x70ea330 thru 0x70f9d62 (0xfa33 bytes) 2024-09-22T01:01:33.467 INFO:teuthology.orchestra.run.smithi002.stdout:2147 read 0x324a34b thru 0x324bf0f (0x1bc5 bytes) 2024-09-22T01:01:33.481 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:33.480+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:33.484 INFO:teuthology.orchestra.run.smithi002.stdout:2148 write 0xbeb91de thru 0xbec1135 (0x7f58 bytes) 2024-09-22T01:01:33.491 INFO:teuthology.orchestra.run.smithi002.stdout:2149 writesame 0x5649686 thru 0x5654d69 data_size 0xa(0xb6e4 bytes) 2024-09-22T01:01:33.494 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:33.494 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:33.509 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:33.516 INFO:teuthology.orchestra.run.smithi002.stdout:2150 read 0xbd94c4f thru 0xbd9d4e0 (0x8892 bytes) 2024-09-22T01:01:33.516 INFO:teuthology.orchestra.run.smithi002.stdout:2151 writesame 0x489bc1e thru 0x48a674f data_size 0x11(0xab32 bytes) 2024-09-22T01:01:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:2155 read 0x86c2b9c thru 0x86cf62a (0xca8f bytes) 2024-09-22T01:01:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:2156 punch from 0x2d0427c to 0x2d0aa34, (0x67b8 bytes) 2024-09-22T01:01:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:33.528 INFO:teuthology.orchestra.run.smithi002.stdout:2158 trunc from 0xbec1136 to 0xa51d307 2024-09-22T01:01:33.540 INFO:teuthology.orchestra.run.smithi002.stdout:2159 read 0x54fcc6d thru 0x550c12b (0xf4bf bytes) 2024-09-22T01:01:33.549 INFO:teuthology.orchestra.run.smithi002.stdout:2160 trunc from 0xa51d307 to 0x3416271 2024-09-22T01:01:33.582 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:33.580+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:33.633 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:33.644 INFO:teuthology.orchestra.run.smithi002.stdout:2162 write 0x3bc3593 thru 0x3bc7e8d (0x48fb bytes) 2024-09-22T01:01:33.646 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:33.646 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:33.648 INFO:teuthology.orchestra.run.smithi002.stdout:2163 read 0x32998a6 thru 0x32a09ae (0x7109 bytes) 2024-09-22T01:01:33.648 INFO:teuthology.orchestra.run.smithi002.stdout:2164 read 0x1574b82 thru 0x1584154 (0xf5d3 bytes) 2024-09-22T01:01:33.649 INFO:teuthology.orchestra.run.smithi002.stdout:2165 read 0x38d1857 thru 0x38df2fc (0xdaa6 bytes) 2024-09-22T01:01:33.649 INFO:teuthology.orchestra.run.smithi002.stdout:2166 read 0x1f43ff6 thru 0x1f4c231 (0x823c bytes) 2024-09-22T01:01:33.651 INFO:teuthology.orchestra.run.smithi002.stdout:2167 punch from 0xdc6efc to 0xdca8f3, (0x39f7 bytes) 2024-09-22T01:01:33.651 INFO:teuthology.orchestra.run.smithi002.stdout:2169 punch from 0x1956ac5 to 0x1957455, (0x990 bytes) 2024-09-22T01:01:33.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:33.679 INFO:teuthology.orchestra.run.smithi002.stdout:2170 write 0xbf185fe thru 0xbf1aecb (0x28ce bytes) 2024-09-22T01:01:33.683 INFO:teuthology.orchestra.run.smithi002.stdout:2171 read 0xa11516 thru 0xa156c1 (0x41ac bytes) 2024-09-22T01:01:33.684 INFO:teuthology.orchestra.run.smithi002.stdout:2172 write 0x3cbdcdb thru 0x3cc3f76 (0x629c bytes) 2024-09-22T01:01:33.686 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:33.686 INFO:teuthology.orchestra.run.smithi002.stdout:2174 write 0x8b0793 thru 0x8c03cd (0xfc3b bytes) 2024-09-22T01:01:33.688 INFO:teuthology.orchestra.run.smithi002.stdout:2178 read 0x8774f26 thru 0x877db27 (0x8c02 bytes) 2024-09-22T01:01:33.688 INFO:teuthology.orchestra.run.smithi002.stdout:2179 punch from 0x956017c to 0x9567a19, (0x789d bytes) 2024-09-22T01:01:33.688 INFO:teuthology.orchestra.run.smithi002.stdout:2180 punch from 0x53a4f26 to 0x53af5d3, (0xa6ad bytes) 2024-09-22T01:01:33.689 INFO:teuthology.orchestra.run.smithi002.stdout:2181 read 0x6c7500c thru 0x6c76862 (0x1857 bytes) 2024-09-22T01:01:33.694 INFO:teuthology.orchestra.run.smithi002.stdout:2183 trunc from 0xbf1aecc to 0x7ce2f08 2024-09-22T01:01:33.710 INFO:teuthology.orchestra.run.smithi002.stdout:2184 trunc from 0x7ce2f08 to 0x8830c8a 2024-09-22T01:01:33.712 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:33.712+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:33.717 INFO:teuthology.orchestra.run.smithi002.stdout:2185 read 0x5bc2c5f thru 0x5bc55ca (0x296c bytes) 2024-09-22T01:01:33.729 INFO:teuthology.orchestra.run.smithi002.stdout:2186 trunc from 0x8830c8a to 0x5862b7 2024-09-22T01:01:33.784 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:33.779+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:33.802 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:33.803 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:33.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:33.872 INFO:teuthology.orchestra.run.smithi002.stdout:2187 punch from 0x4fa2b4 to 0x4fd47a, (0x31c6 bytes) 2024-09-22T01:01:33.873 INFO:teuthology.orchestra.run.smithi002.stdout:2188 read 0x4b8bd5 thru 0x4c0f08 (0x8334 bytes) 2024-09-22T01:01:33.904 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:33.904+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:33.922 INFO:teuthology.orchestra.run.smithi002.stdout:2189 writesame 0xd84d28b thru 0xd84ecff data_size 0x1(0x1a75 bytes) 2024-09-22T01:01:33.923 INFO:teuthology.orchestra.run.smithi002.stdout:2190 read 0xa500a04 thru 0xa50afa3 (0xa5a0 bytes) 2024-09-22T01:01:33.923 INFO:teuthology.orchestra.run.smithi002.stdout:2191 write 0x81fda8 thru 0x823e9e (0x40f7 bytes) 2024-09-22T01:01:33.930 INFO:teuthology.orchestra.run.smithi002.stdout:2192 punch from 0x1bdea1c to 0x1bed532, (0xeb16 bytes) 2024-09-22T01:01:33.930 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:33.930 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:33.930 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:33.930 INFO:teuthology.orchestra.run.smithi002.stdout:2199 writesame 0xb2847d7 thru 0xb286543 data_size 0x1f(0x1d6d bytes) 2024-09-22T01:01:33.935 INFO:teuthology.orchestra.run.smithi002.stdout:2200 writesame 0x5fd8f22 thru 0x5fe2d4d data_size 0x35(0x9e2c bytes) 2024-09-22T01:01:33.943 INFO:teuthology.orchestra.run.smithi002.stdout:2201 write 0xd1c51d5 thru 0xd1cc238 (0x7064 bytes) 2024-09-22T01:01:33.949 INFO:teuthology.orchestra.run.smithi002.stdout:2203 read 0x938ae41 thru 0x9395518 (0xa6d8 bytes) 2024-09-22T01:01:33.950 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:33.951 INFO:teuthology.orchestra.run.smithi002.stdout:2206 write 0x2140e51 thru 0x21415a4 (0x754 bytes) 2024-09-22T01:01:33.954 INFO:teuthology.orchestra.run.smithi002.stdout:2207 read 0x3c631fc thru 0x3c71b9c (0xe9a1 bytes) 2024-09-22T01:01:33.973 INFO:teuthology.orchestra.run.smithi002.stdout:2209 trunc from 0xd84ed00 to 0x6a0cb88 2024-09-22T01:01:33.982 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:33.982 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:33.985 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:33.983+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:33.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:34.017 INFO:teuthology.orchestra.run.smithi002.stdout:2211 trunc from 0x6a0cb88 to 0xba9f4c 2024-09-22T01:01:34.035 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.4 to 1 (10000) 2024-09-22T01:01:34.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-22T01:01:34.068 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:01:34.081 INFO:teuthology.orchestra.run.smithi002.stdout:2212 writesame 0xe226415 thru 0xe227ee5 data_size 0x1(0x1ad1 bytes) 2024-09-22T01:01:34.081 INFO:teuthology.orchestra.run.smithi002.stdout:2213 punch from 0xd707e85 to 0xd70895b, (0xad6 bytes) 2024-09-22T01:01:34.081 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:34.087 INFO:teuthology.orchestra.run.smithi002.stdout:2216 trunc from 0xe227ee6 to 0xb6d5064 2024-09-22T01:01:34.088 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:34.083+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:34.097 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:34.097 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:34.101 INFO:teuthology.orchestra.run.smithi002.stdout:2220 trunc from 0xb6d5064 to 0x91918d4 2024-09-22T01:01:34.113 INFO:teuthology.orchestra.run.smithi002.stdout:2221 trunc from 0x91918d4 to 0x4edfda8 2024-09-22T01:01:34.141 INFO:teuthology.orchestra.run.smithi002.stdout:2224 writesame 0xe4add7d thru 0xe4baf9a data_size 0x1e(0xd21e bytes) 2024-09-22T01:01:34.141 INFO:teuthology.orchestra.run.smithi002.stdout:2225 punch from 0x65d5b1b to 0x65e5106, (0xf5eb bytes) 2024-09-22T01:01:34.152 INFO:teuthology.orchestra.run.smithi002.stdout:2226 trunc from 0xe4baf9b to 0x5b73a77 2024-09-22T01:01:34.160 INFO:teuthology.orchestra.run.smithi002.stdout:2227 read 0x3130b99 thru 0x313e842 (0xdcaa bytes) 2024-09-22T01:01:34.163 INFO:teuthology.orchestra.run.smithi002.stdout:2229 punch from 0x5c28ac to 0x5d1d42, (0xf496 bytes) 2024-09-22T01:01:34.163 INFO:teuthology.orchestra.run.smithi002.stdout:2231 punch from 0x47e105e to 0x47e8132, (0x70d4 bytes) 2024-09-22T01:01:34.163 INFO:teuthology.orchestra.run.smithi002.stdout:2232 read 0x43c555c thru 0x43cc960 (0x7405 bytes) 2024-09-22T01:01:34.163 INFO:teuthology.orchestra.run.smithi002.stdout:2233 punch from 0x5905d29 to 0x590b3c7, (0x569e bytes) 2024-09-22T01:01:34.163 INFO:teuthology.orchestra.run.smithi002.stdout:2234 trunc from 0x5b73a77 to 0x5fc146e 2024-09-22T01:01:34.165 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:34.165 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:34.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:34.184 INFO:teuthology.orchestra.run.smithi002.stdout:2235 write 0xb45d03f thru 0xb46ba79 (0xea3b bytes) 2024-09-22T01:01:34.186 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:34.183+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:34.200 INFO:teuthology.orchestra.run.smithi002.stdout:2236 trunc from 0xb46ba7a to 0x314109d 2024-09-22T01:01:34.215 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:34.227 INFO:teuthology.orchestra.run.smithi002.stdout:2238 write 0x7790872 thru 0x77979f5 (0x7184 bytes) 2024-09-22T01:01:34.231 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:34.232 INFO:teuthology.orchestra.run.smithi002.stdout:2240 punch from 0x777b10b to 0x777b55a, (0x44f bytes) 2024-09-22T01:01:34.234 INFO:teuthology.orchestra.run.smithi002.stdout:2241 trunc from 0x77979f6 to 0x98ded30 2024-09-22T01:01:34.241 INFO:teuthology.orchestra.run.smithi002.stdout:2242 read 0x20a29f3 thru 0x20b2324 (0xf932 bytes) 2024-09-22T01:01:34.245 INFO:teuthology.orchestra.run.smithi002.stdout:2243 trunc from 0x98ded30 to 0xcba5a5a 2024-09-22T01:01:34.252 INFO:teuthology.orchestra.run.smithi002.stdout:2244 punch from 0xc6bddcb to 0xc6bfb52, (0x1d87 bytes) 2024-09-22T01:01:34.252 INFO:teuthology.orchestra.run.smithi002.stdout:2249 punch from 0x9a6f2f1 to 0x9a7e9e1, (0xf6f0 bytes) 2024-09-22T01:01:34.257 INFO:teuthology.orchestra.run.smithi002.stdout:2251 trunc from 0xcba5a5a to 0x80d8247 2024-09-22T01:01:34.264 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:34.264 INFO:teuthology.orchestra.run.smithi002.stdout:2253 read 0x30d5084 thru 0x30d78fa (0x2877 bytes) 2024-09-22T01:01:34.265 INFO:teuthology.orchestra.run.smithi002.stdout:2254 punch from 0x5608123 to 0x5615eff, (0xdddc bytes) 2024-09-22T01:01:34.276 INFO:teuthology.orchestra.run.smithi002.stdout:2255 write 0xb7a05ff thru 0xb7a55df (0x4fe1 bytes) 2024-09-22T01:01:34.281 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:34.281 INFO:teuthology.orchestra.run.smithi002.stdout:2259 write 0x48a7e6f thru 0x48ae5b7 (0x6749 bytes) 2024-09-22T01:01:34.285 INFO:teuthology.orchestra.run.smithi002.stdout:2261 read 0x7b69f44 thru 0x7b6ea42 (0x4aff bytes) 2024-09-22T01:01:34.285 INFO:teuthology.orchestra.run.smithi002.stdout:2263 read 0x8b2070d thru 0x8b30066 (0xf95a bytes) 2024-09-22T01:01:34.285 INFO:teuthology.orchestra.run.smithi002.stdout:2264 read 0x166b580 thru 0x1675773 (0xa1f4 bytes) 2024-09-22T01:01:34.285 INFO:teuthology.orchestra.run.smithi002.stdout:2265 write 0x3cea61b thru 0x3cf1262 (0x6c48 bytes) 2024-09-22T01:01:34.287 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:34.283+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:34.296 INFO:teuthology.orchestra.run.smithi002.stdout:2266 trunc from 0xb7a55e0 to 0x5e4e103 2024-09-22T01:01:34.309 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:34.309 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:34.312 INFO:teuthology.orchestra.run.smithi002.stdout:2267 clone 23 order 22 su 2097152 sc 15 2024-09-22T01:01:34.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:34.388 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:34.388+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:34.441 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:34.441 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:34.456 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:34.489 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:34.487+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:34.590 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:34.588+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:34.625 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:34.625 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:34.638 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:34.724 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:34.724+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:34.792 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:34.787+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:34.805 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:34.805 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:34.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:34.871 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:34.918 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:34.915+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:34.982 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:34.982 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:34.992 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:34.987+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:34.996 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:35.028 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.5 to 1 (10000) 2024-09-22T01:01:35.031 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:35.031 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:35.031 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:01:35.055 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:35.055 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:35.056 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-22T01:01:35.072 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:35.072 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:35.091 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:35.094 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:35.092+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:35.111 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:35.112 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:35.127 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:35.127 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:35.132 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:35.132 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:35.143 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:35.151 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:35.151 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:35.162 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone22 from 0x5e4e103 (overlap 0x5862b7) to 0x0 2024-09-22T01:01:35.174 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:35.174 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:35.237 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:35.235+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-22T01:01:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:2269 write 0x4cac4da thru 0x4cb8eea (0xca11 bytes) 2024-09-22T01:01:35.291 INFO:teuthology.orchestra.run.smithi002.stdout:2270 trunc from 0x5e4e103 to 0xb56c4ab 2024-09-22T01:01:35.295 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:35.292+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:35.302 INFO:teuthology.orchestra.run.smithi002.stdout:2271 trunc from 0xb56c4ab to 0x97f8b62 2024-09-22T01:01:35.311 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:35.311 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:35.311 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:35.311 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:35.312 INFO:teuthology.orchestra.run.smithi002.stdout:2272 writesame 0xc42fbd thru 0xc52f12 data_size 0x62(0xff56 bytes) 2024-09-22T01:01:35.319 INFO:teuthology.orchestra.run.smithi002.stdout:2273 trunc from 0x97f8b62 to 0xa3fce73 2024-09-22T01:01:35.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:01:35.329 INFO:teuthology.orchestra.run.smithi002.stdout:2274 writesame 0x35d9bff thru 0x35dd1ba data_size 0x26(0x35bc bytes) 2024-09-22T01:01:35.336 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:35.340 INFO:teuthology.orchestra.run.smithi002.stdout:2276 trunc from 0xa3fce73 to 0xc202ab9 2024-09-22T01:01:35.350 INFO:teuthology.orchestra.run.smithi002.stdout:2278 punch from 0x8c2e529 to 0x8c3253c, (0x4013 bytes) 2024-09-22T01:01:35.359 INFO:teuthology.orchestra.run.smithi002.stdout:2279 writesame 0xc2f79f6 thru 0xc2fd81c data_size 0x1(0x5e27 bytes) 2024-09-22T01:01:35.359 INFO:teuthology.orchestra.run.smithi002.stdout:2280 read 0x8681c57 thru 0x868b74f (0x9af9 bytes) 2024-09-22T01:01:35.365 INFO:teuthology.orchestra.run.smithi002.stdout:2281 trunc from 0xc2fd81d to 0x930a4d0 2024-09-22T01:01:35.376 INFO:teuthology.orchestra.run.smithi002.stdout:2282 writesame 0x7cfe31c thru 0x7d02103 data_size 0x1c(0x3de8 bytes) 2024-09-22T01:01:35.381 INFO:teuthology.orchestra.run.smithi002.stdout:2283 write 0x4a5a0b5 thru 0x4a5dae7 (0x3a33 bytes) 2024-09-22T01:01:35.385 INFO:teuthology.orchestra.run.smithi002.stdout:2284 write 0x8df11d2 thru 0x8df3fc7 (0x2df6 bytes) 2024-09-22T01:01:35.396 INFO:teuthology.orchestra.run.smithi002.stdout:2285 write 0x9707bc2 thru 0x970b9ee (0x3e2d bytes) 2024-09-22T01:01:35.400 INFO:teuthology.orchestra.run.smithi002.stdout:2287 read 0x97069d thru 0x97e33f (0xdca3 bytes) 2024-09-22T01:01:35.401 INFO:teuthology.orchestra.run.smithi002.stdout:2289 write 0x78af11a thru 0x78b7748 (0x862f bytes) 2024-09-22T01:01:35.403 INFO:teuthology.orchestra.run.smithi002.stdout:2290 read 0x684f5a3 thru 0x685f223 (0xfc81 bytes) 2024-09-22T01:01:35.404 INFO:teuthology.orchestra.run.smithi002.stdout:2293 punch from 0x4a4d526 to 0x4a56efd, (0x99d7 bytes) 2024-09-22T01:01:35.404 INFO:teuthology.orchestra.run.smithi002.stdout:2295 read 0x26cddac thru 0x26d3d70 (0x5fc5 bytes) 2024-09-22T01:01:35.404 INFO:teuthology.orchestra.run.smithi002.stdout:2297 writesame 0x5a924b1 thru 0x5a968cf data_size 0x1(0x441f bytes) 2024-09-22T01:01:35.416 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:35.416+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:35.427 INFO:teuthology.orchestra.run.smithi002.stdout:2300 punch from 0x94bcab3 to 0x94c39dd, (0x6f2a bytes) 2024-09-22T01:01:35.437 INFO:teuthology.orchestra.run.smithi002.stdout:2301 write 0xa6394f1 thru 0xa63beec (0x29fc bytes) 2024-09-22T01:01:35.458 INFO:teuthology.orchestra.run.smithi002.stdout:2303 trunc from 0xa63beed to 0x2d0a619 2024-09-22T01:01:35.496 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:35.491+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:35.501 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:35.502 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:35.502 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:35.502 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:35.502 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:35.514 INFO:teuthology.orchestra.run.smithi002.stdout:2305 writesame 0x48c2f3a thru 0x48d1af7 data_size 0x19(0xebbe bytes) 2024-09-22T01:01:35.515 INFO:teuthology.orchestra.run.smithi002.stdout:2306 punch from 0xe869df to 0xe944e7, (0xdb08 bytes) 2024-09-22T01:01:35.515 INFO:teuthology.orchestra.run.smithi002.stdout:2307 read 0x2af008b thru 0x2af9439 (0x93af bytes) 2024-09-22T01:01:35.515 INFO:teuthology.orchestra.run.smithi002.stdout:2309 write 0x46a85bc thru 0x46af802 (0x7247 bytes) 2024-09-22T01:01:35.517 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:01:35.522 INFO:teuthology.orchestra.run.smithi002.stdout:2310 punch from 0x1b76b6d to 0x1b859fc, (0xee8f bytes) 2024-09-22T01:01:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:2312 trunc from 0x48d1af8 to 0x92ee817 2024-09-22T01:01:35.545 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:35.545 INFO:teuthology.orchestra.run.smithi002.stdout:2314 write 0x54f973a thru 0x54fae0b (0x16d2 bytes) 2024-09-22T01:01:35.552 INFO:teuthology.orchestra.run.smithi002.stdout:2315 read 0x6002651 thru 0x600dbe6 (0xb596 bytes) 2024-09-22T01:01:35.570 INFO:teuthology.orchestra.run.smithi002.stdout:2317 writesame 0xcc89eac thru 0xcc8c2c4 data_size 0x1(0x2419 bytes) 2024-09-22T01:01:35.570 INFO:teuthology.orchestra.run.smithi002.stdout:2318 write 0x6fb149e thru 0x6fbf33f (0xdea2 bytes) 2024-09-22T01:01:35.585 INFO:teuthology.orchestra.run.smithi002.stdout:2321 write 0xcd25579 thru 0xcd29dbc (0x4844 bytes) 2024-09-22T01:01:35.591 INFO:teuthology.orchestra.run.smithi002.stdout:2322 writesame 0x26b80c4 thru 0x26bf25e data_size 0x1(0x719b bytes) 2024-09-22T01:01:35.597 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:35.591+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:35.636 INFO:teuthology.orchestra.run.smithi002.stdout:2323 punch from 0x216e0db to 0x217b60e, (0xd533 bytes) 2024-09-22T01:01:35.636 INFO:teuthology.orchestra.run.smithi002.stdout:2324 read 0x7e7ff4d thru 0x7e84390 (0x4444 bytes) 2024-09-22T01:01:35.636 INFO:teuthology.orchestra.run.smithi002.stdout:2326 read 0xa5601e5 thru 0xa564982 (0x479e bytes) 2024-09-22T01:01:35.636 INFO:teuthology.orchestra.run.smithi002.stdout:2327 write 0x880ba9f thru 0x8819217 (0xd779 bytes) 2024-09-22T01:01:35.643 INFO:teuthology.orchestra.run.smithi002.stdout:2331 writesame 0x270a00b thru 0x27170e2 data_size 0x52(0xd0d8 bytes) 2024-09-22T01:01:35.647 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:35.647 INFO:teuthology.orchestra.run.smithi002.stdout:2335 read 0x2e7c2ce thru 0x2e88a2c (0xc75f bytes) 2024-09-22T01:01:35.648 INFO:teuthology.orchestra.run.smithi002.stdout:2337 read 0x30f22b7 thru 0x30f4349 (0x2093 bytes) 2024-09-22T01:01:35.649 INFO:teuthology.orchestra.run.smithi002.stdout:2338 punch from 0x1d8fbcd to 0x1d9e62c, (0xea5f bytes) 2024-09-22T01:01:35.649 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:35.649 INFO:teuthology.orchestra.run.smithi002.stdout:2340 write 0xf30a thru 0x15e4e (0x6b45 bytes) 2024-09-22T01:01:35.675 INFO:teuthology.orchestra.run.smithi002.stdout:2341 writesame 0x163fd4b thru 0x1640acf data_size 0x1(0xd85 bytes) 2024-09-22T01:01:35.681 INFO:teuthology.orchestra.run.smithi002.stdout:2342 write 0xab9b3fd thru 0xab9fd5b (0x495f bytes) 2024-09-22T01:01:35.688 INFO:teuthology.orchestra.run.smithi002.stdout:2344 read 0x87d387e thru 0x87dd5ee (0x9d71 bytes) 2024-09-22T01:01:35.688 INFO:teuthology.orchestra.run.smithi002.stdout:2345 read 0x6742a85 thru 0x6749ade (0x705a bytes) 2024-09-22T01:01:35.688 INFO:teuthology.orchestra.run.smithi002.stdout:2346 read 0x54cb922 thru 0x54db034 (0xf713 bytes) 2024-09-22T01:01:35.689 INFO:teuthology.orchestra.run.smithi002.stdout:2347 writesame 0xaaade62 thru 0xaab18eb data_size 0x3b(0x3a8a bytes) 2024-09-22T01:01:35.692 INFO:teuthology.orchestra.run.smithi002.stdout:2348 write 0x95eb07c thru 0x95ecdb6 (0x1d3b bytes) 2024-09-22T01:01:35.698 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:35.698 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:35.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:35.698 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:35.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.698 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:head, version: 42'67, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:head@42'67, size: 1078481, copy_subset: [(0, 1078481)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:35.698 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.799438+0000", 2024-09-22T01:01:35.698 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.8960415069999996, 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047812239999999997, 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.799438+0000", 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.799438+0000", 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.799440+0000", 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.466e-06 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801127+0000", 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016877400000000001 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801165+0000", 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7718e-05 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801193+0000", 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8189000000000001e-05 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801339+0000", 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014596699999999999 2024-09-22T01:01:35.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.801359+0000", 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.804219+0000", 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002860521 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e, version: 42'66, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e@42'66, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.810104+0000", 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.8853754289999998, 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041628430000000003, 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.810104+0000", 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.810104+0000", 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.810105+0000", 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8199999999999998e-07 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811265+0000", 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011604230000000001 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811302+0000", 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6174999999999998e-05 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811329+0000", 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7406000000000002e-05 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811569+0000", 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024011599999999999 2024-09-22T01:01:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.811605+0000", 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5336000000000002e-05 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.814267+0000", 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026625049999999999 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.16 53/52 [PushOp(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e, version: 42'66, data_included: [2097152~28672,2138112~28672,2179072~28672,2220032~28672,2260992~28672,2301952~28672,2342912~344064], data_size: 516096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:69fc1cdd:::rbd_data.117c6f6b4a3e.000000000000000a:e@42'66, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3670016, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.824091+0000", 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.8713888560000003, 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042729949999999999, 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.707 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824091+0000", 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824091+0000", 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824093+0000", 2024-09-22T01:01:35.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.79e-06 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824687+0000", 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059397200000000001 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824733+0000", 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6870999999999999e-05 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824769+0000", 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5982999999999998e-05 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.824967+0000", 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000197448 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.825005+0000", 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8217999999999999e-05 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.828364+0000", 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033587130000000001 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.c 53/52 [PushOp(3:33db21cf:::rbd_data.117ccb654ee9.0000000000000001:7, version: 34'21, data_included: [0~524288,1258536~36357], data_size: 560645, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:33db21cf:::rbd_data.117ccb654ee9.0000000000000001:7@34'21, size: 1294893, copy_subset: [(0, 1294893)], clone_subset: {}, snapset: 7=[]:{7: [7]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1294893, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.878858+0000", 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.8166215750000001, 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004942475, 2024-09-22T01:01:35.711 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.878858+0000", 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.878858+0000", 2024-09-22T01:01:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.878860+0000", 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.68e-06 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880689+0000", 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018293059999999999 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880712+0000", 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3155000000000001e-05 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880729+0000", 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7142999999999998e-05 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880891+0000", 2024-09-22T01:01:35.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000161705 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.880915+0000", 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3903000000000001e-05 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.883801+0000", 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028855830000000002 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPush(3.d 53/52 [PushOp(3:b666f596:::rbd_data.117c6f6b4a3e.0000000000000007:e, version: 42'46, data_included: [2097152~20480,2129920~28672,2170880~28672,2211840~28672,2252800~28672,2293760~28672,2334720~352256], data_size: 516096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b666f596:::rbd_data.117c6f6b4a3e.0000000000000007:e@42'46, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3670016, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:27.892918+0000", 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.8025614149999996, 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039331339999999996, 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.892918+0000", 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.892918+0000", 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.892920+0000", 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3859999999999999e-06 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893516+0000", 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059648100000000005 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893558+0000", 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1544000000000001e-05 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893588+0000", 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0527000000000001e-05 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893811+0000", 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022269600000000001 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.893843+0000", 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2001000000000001e-05 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:27.896851+0000", 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003008499 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40184 3.16 3:6a202c98:::rbd_data.117c35b6a0e9.000000000000006b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.641544+0000", 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.0539352700000002, 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089830840000000006, 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40184 2024-09-22T01:01:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.641544+0000", 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.641544+0000", 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.641546+0000", 2024-09-22T01:01:35.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8029999999999999e-06 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.641555+0000", 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0790000000000007e-06 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.641579+0000", 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3693000000000002e-05 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.641592+0000", 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-05 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.641654+0000", 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2447999999999996e-05 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.641704+0000", 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9681000000000001e-05 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.642260+0000", 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055607599999999997 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.645985+0000", 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037251150000000002 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.650245+0000", 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004259723 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.650404+0000", 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015916699999999999 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.650527+0000", 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012369900000000001 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40222 3.16 3:6a202c98:::rbd_data.117c35b6a0e9.000000000000006b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.725 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.718707+0000", 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9767730439999998, 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049932609999999997, 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40222 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718707+0000", 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718707+0000", 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718709+0000", 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4559999999999999e-06 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718724+0000", 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4629e-05 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718765+0000", 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1331999999999997e-05 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718787+0000", 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1670000000000001e-05 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718947+0000", 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000159887 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.719143+0000", 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019690500000000001 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.719427+0000", 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028397600000000002 2024-09-22T01:01:35.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723353+0000", 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039252590000000004 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723469+0000", 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116659 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723595+0000", 2024-09-22T01:01:35.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000125275 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723700+0000", 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105213 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40228 3.c 3:3114fff0:::rbd_data.117c35b6a0e9.000000000000007c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.731 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.718840+0000", 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9766393190000002, 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048790589999999998, 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40228 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718840+0000", 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718840+0000", 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718842+0000", 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.449e-06 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718850+0000", 2024-09-22T01:01:35.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3599999999999996e-06 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718872+0000", 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2052000000000002e-05 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718888+0000", 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6277e-05 2024-09-22T01:01:35.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.719043+0000", 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000154654 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.719246+0000", 2024-09-22T01:01:35.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020249499999999999 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.719521+0000", 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027524299999999999 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.721338+0000", 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001816715 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723466+0000", 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021282699999999998 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723603+0000", 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013748100000000001 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723719+0000", 2024-09-22T01:01:35.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116063 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40229 3.1c 3:3b146e3d:::rbd_data.117c35b6a0e9.00000000000000a5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.718931+0000", 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9765481400000002, 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050513770000000001, 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40229 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718931+0000", 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718931+0000", 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718933+0000", 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.393e-06 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718940+0000", 2024-09-22T01:01:35.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.34e-06 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718962+0000", 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2252999999999999e-05 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.718979+0000", 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6271999999999999e-05 2024-09-22T01:01:35.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.719085+0000", 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000106173 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.719271+0000", 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018616600000000001 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.719521+0000", 2024-09-22T01:01:35.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024975099999999999 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723073+0000", 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035520690000000001 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723747+0000", 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067448499999999995 2024-09-22T01:01:35.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723880+0000", 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132695 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723983+0000", 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010278 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40236 3.16 3:6a86d430:::rbd_data.117c35b6a0e9.000000000000015f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.723635+0000", 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9718449800000002, 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067564180000000001, 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40236 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723635+0000", 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723635+0000", 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723637+0000", 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0530000000000001e-06 2024-09-22T01:01:35.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723647+0000", 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8819999999999996e-06 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723669+0000", 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723689+0000", 2024-09-22T01:01:35.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9585999999999998e-05 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.723833+0000", 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143614 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.724059+0000", 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022618599999999999 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.724355+0000", 2024-09-22T01:01:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029574899999999999 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.727806+0000", 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034514260000000001 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.730115+0000", 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002308919 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.730257+0000", 2024-09-22T01:01:35.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014160799999999999 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.730391+0000", 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000134492 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40246 3.16 3:6b3c190f:::rbd_data.117c35b6a0e9.00000000000001da:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.728448+0000", 2024-09-22T01:01:35.750 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.967031532, 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039989659999999996, 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40246 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.728448+0000", 2024-09-22T01:01:35.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.728448+0000", 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.728450+0000", 2024-09-22T01:01:35.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1830000000000001e-06 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.728460+0000", 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4930000000000004e-06 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.728487+0000", 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.728512+0000", 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5018999999999999e-05 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.728845+0000", 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033216099999999998 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.729077+0000", 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000232148 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.729312+0000", 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000235677 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.730647+0000", 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001334443 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.732265+0000", 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001617643 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.732358+0000", 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3166999999999998e-05 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.732447+0000", 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9322999999999998e-05 2024-09-22T01:01:35.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40361 3.5 3:a15031b0:::rbd_data.117c162c7df4.000000000000000f:16 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.738878+0000", 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9566019589999999, 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003942325, 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40361 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738878+0000", 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738878+0000", 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738878+0000", 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738883+0000", 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2690000000000001e-06 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738892+0000", 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5559999999999996e-06 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738899+0000", 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.742239+0000", 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033404279999999999 2024-09-22T01:01:35.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.742513+0000", 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027374000000000002 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.742820+0000", 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030683799999999998 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40409 3.5 3:a15031b0:::rbd_data.117c162c7df4.000000000000000f:16 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.738912+0000", 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9565676229999998, 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045600179999999999, 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40409 2024-09-22T01:01:35.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738912+0000", 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738912+0000", 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738913+0000", 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3300000000000002e-07 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738916+0000", 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738925+0000", 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2399999999999996e-06 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738932+0000", 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.703e-06 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.742929+0000", 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039972059999999997 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.743182+0000", 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025367400000000001 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.743472+0000", 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028957899999999999 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40410 3.5 3:a15031b0:::rbd_data.117c162c7df4.000000000000000f:16 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:31.738944+0000", 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9565355919999998, 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051441059999999999, 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.766 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40410 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738944+0000", 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738944+0000", 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738945+0000", 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738948+0000", 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.96e-06 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738958+0000", 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0866999999999999e-05 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.738965+0000", 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9759999999999998e-06 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.743549+0000", 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045833540000000004 2024-09-22T01:01:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.743804+0000", 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025476999999999997 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:31.744088+0000", 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028455899999999998 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.770 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40590 3.1c 3:3a560c54:::rbd_data.117c35b6a0e9.0000000000000003:head [stat,delete] snapc 17=[17] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.182679+0000", 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5128003059999999, 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044632999999999999, 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40590 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182679+0000", 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182679+0000", 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182682+0000", 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4830000000000002e-06 2024-09-22T01:01:35.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182694+0000", 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2510000000000001e-05 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182727+0000", 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2398999999999999e-05 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182744+0000", 2024-09-22T01:01:35.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7334999999999999e-05 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182907+0000", 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016337099999999999 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.183253+0000", 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000345645 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.187143+0000", 2024-09-22T01:01:35.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038895570000000001 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40595 3.1c 3:39e1c0ae:::rbd_data.117c35b6a0e9.0000000000000008:head [stat,delete] snapc 17=[17] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.182792+0000", 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5126877429999999, 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005325682, 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40595 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182792+0000", 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182792+0000", 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182793+0000", 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.336e-06 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182801+0000", 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2660000000000001e-06 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182828+0000", 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6455999999999999e-05 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.182844+0000", 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5832999999999999e-05 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.184250+0000", 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00140661 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.184576+0000", 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00032582500000000002 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.188118+0000", 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035413559999999998 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41097 3.1c 3:39e1c0ae:::rbd_data.117c35b6a0e9.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.366401+0000", 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.3290786570000002, 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066492269999999997, 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41097 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.366401+0000", 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.366401+0000", 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.366414+0000", 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3008e-05 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367021+0000", 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000606738 2024-09-22T01:01:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367050+0000", 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9598999999999999e-05 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367066+0000", 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5271e-05 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367203+0000", 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013696699999999999 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367416+0000", 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213489 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.368002+0000", 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058609299999999999 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.372451+0000", 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044485339999999996 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.372741+0000", 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029067900000000001 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.372934+0000", 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019261699999999999 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.373050+0000", 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011623199999999999 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41098 3.1c 3:39e1c0ae:::rbd_data.117c35b6a0e9.0000000000000008:head [delete] snapc 17=[17] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.367100+0000", 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.3283796300000001, 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062184980000000003, 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41098 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367100+0000", 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367100+0000", 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367102+0000", 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.584e-06 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367108+0000", 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2559999999999999e-06 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367124+0000", 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5707000000000001e-05 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.367144+0000", 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0356e-05 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.369929+0000", 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027853819999999999 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.370131+0000", 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020215700000000001 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.370556+0000", 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000424774 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.373110+0000", 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025535509999999998 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.373184+0000", 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4284999999999998e-05 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.373263+0000", 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8818000000000003e-05 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.373318+0000", 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5627999999999997e-05 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41114 3.c 3:376adb3c:::rbd_data.117c35b6a0e9.00000000000002fc:head [delete] snapc 17=[17] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.491500+0000", 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.203979446, 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062286950000000002, 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41114 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.491500+0000", 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.491500+0000", 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.491504+0000", 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9689999999999996e-06 2024-09-22T01:01:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.491521+0000", 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6473e-05 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.491568+0000", 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7321999999999999e-05 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.491596+0000", 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7650000000000001e-05 2024-09-22T01:01:35.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.491765+0000", 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000169805 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.492018+0000", 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025235500000000001 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.492476+0000", 2024-09-22T01:01:35.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00045844400000000002 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.494559+0000", 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002082745 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.497477+0000", 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029178440000000002 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.497608+0000", 2024-09-22T01:01:35.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013157800000000001 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.497729+0000", 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012051 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41246 3.1c 3:3ca521b8:::rbd_data.117ce0697193.000000000000004f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.656715+0000", 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.0387641780000001, 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006308091, 2024-09-22T01:01:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41246 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.656715+0000", 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.656715+0000", 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.656718+0000", 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5629999999999999e-06 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.656730+0000", 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2163e-05 2024-09-22T01:01:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.656760+0000", 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0071999999999999e-05 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.656780+0000", 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0132999999999998e-05 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.656899+0000", 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011880899999999999 2024-09-22T01:01:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.657116+0000", 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021670599999999999 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.657380+0000", 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000263634 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.658888+0000", 2024-09-22T01:01:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001508496 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.662708+0000", 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038195299999999998 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.662894+0000", 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000186083 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.663024+0000", 2024-09-22T01:01:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012990199999999999 2024-09-22T01:01:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:35.802 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:35.803 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:35.695+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:35.804 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:35.800+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:35.804 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:2349 trunc from 0xcd29dbd to 0x9063f86 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:2352 write 0xd43010f thru 0xd43041a (0x30c bytes) 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:2353 read 0x33de641 thru 0x33e75b7 (0x8f77 bytes) 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:2355 writesame 0x4f52bd9 thru 0x4f56ffe data_size 0x1a(0x4426 bytes) 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:2356 write 0xe26cde4 thru 0xe27436c (0x7589 bytes) 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:2358 write 0x89e8cbf thru 0x89ea713 (0x1a55 bytes) 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:2359 write 0x79aca83 thru 0x79b510e (0x868c bytes) 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:2363 read 0x6c975de thru 0x6ca4462 (0xce85 bytes) 2024-09-22T01:01:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:2364 punch from 0x5543532 to 0x5543b80, (0x64e bytes) 2024-09-22T01:01:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:2365 write 0x40c0bf thru 0x416913 (0xa855 bytes) 2024-09-22T01:01:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:2367 punch from 0xb94c20b to 0xb95a992, (0xe787 bytes) 2024-09-22T01:01:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:2370 writesame 0x55a735b thru 0x55b3ae3 data_size 0x1(0xc789 bytes) 2024-09-22T01:01:35.878 INFO:teuthology.orchestra.run.smithi002.stdout:2371 read 0x667692e thru 0x668601e (0xf6f1 bytes) 2024-09-22T01:01:35.878 INFO:teuthology.orchestra.run.smithi002.stdout:2373 read 0xa6cfae7 thru 0xa6de45b (0xe975 bytes) 2024-09-22T01:01:35.879 INFO:teuthology.orchestra.run.smithi002.stdout:2374 punch from 0x1fa3355 to 0x1fb120f, (0xdeba bytes) 2024-09-22T01:01:35.879 INFO:teuthology.orchestra.run.smithi002.stdout:2376 read 0x9bdd480 thru 0x9be40fb (0x6c7c bytes) 2024-09-22T01:01:35.879 INFO:teuthology.orchestra.run.smithi002.stdout:2377 write 0x513d074 thru 0x513fd57 (0x2ce4 bytes) 2024-09-22T01:01:35.881 INFO:teuthology.orchestra.run.smithi002.stdout:2379 write 0x451d097 thru 0x452b66b (0xe5d5 bytes) 2024-09-22T01:01:35.904 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:35.899+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:35.909 INFO:teuthology.orchestra.run.smithi002.stdout:2380 trunc from 0xe27436d to 0x1e7fe16 2024-09-22T01:01:35.950 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:35.951 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:35.951 INFO:teuthology.orchestra.run.smithi002.stdout:2381 punch from 0x3b5e5 to 0x3be49, (0x864 bytes) 2024-09-22T01:01:35.951 INFO:teuthology.orchestra.run.smithi002.stdout:2382 read 0x107718 thru 0x1140cc (0xc9b5 bytes) 2024-09-22T01:01:35.952 INFO:teuthology.orchestra.run.smithi002.stdout:2384 read 0x1ba435a thru 0x1bb1b4e (0xd7f5 bytes) 2024-09-22T01:01:35.953 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:35.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:35.990 INFO:teuthology.orchestra.run.smithi002.stdout:2386 write 0xc6c4b7f thru 0xc6c57bd (0xc3f bytes) 2024-09-22T01:01:35.997 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:36.004 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:36.004+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:36.012 INFO:teuthology.orchestra.run.smithi002.stdout:2388 trunc from 0xc6c57be to 0x631e8fa 2024-09-22T01:01:36.034 INFO:teuthology.orchestra.run.smithi002.stdout:2389 punch from 0x54b0a03 to 0x54b82ca, (0x78c7 bytes) 2024-09-22T01:01:36.035 INFO:teuthology.orchestra.run.smithi002.stdout:2390 read 0x83aa97 thru 0x83d721 (0x2c8b bytes) 2024-09-22T01:01:36.035 INFO:teuthology.orchestra.run.smithi002.stdout:2392 read 0x967a29 thru 0x967f00 (0x4d8 bytes) 2024-09-22T01:01:36.035 INFO:teuthology.orchestra.run.smithi002.stdout:2395 read 0x4b6006 thru 0x4c540e (0xf409 bytes) 2024-09-22T01:01:36.037 INFO:teuthology.orchestra.run.smithi002.stdout:2397 punch from 0x500a65 to 0x50513f, (0x46da bytes) 2024-09-22T01:01:36.037 INFO:teuthology.orchestra.run.smithi002.stdout:2399 read 0x6023e87 thru 0x6026485 (0x25ff bytes) 2024-09-22T01:01:36.037 INFO:teuthology.orchestra.run.smithi002.stdout:2400 read 0x5a7215c thru 0x5a734a6 (0x134b bytes) 2024-09-22T01:01:36.044 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.6 to 1 (10000) 2024-09-22T01:01:36.069 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-22T01:01:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:2402 write 0xcd648a1 thru 0xcd67a69 (0x31c9 bytes) 2024-09-22T01:01:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:2403 punch from 0x75db284 to 0x75e4443, (0x91bf bytes) 2024-09-22T01:01:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:2404 read 0x54c0be4 thru 0x54c6768 (0x5b85 bytes) 2024-09-22T01:01:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:2405 trunc from 0xcd67a6a to 0x9416e75 2024-09-22T01:01:36.105 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:36.104+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:36.111 INFO:teuthology.orchestra.run.smithi002.stdout:2406 read 0x8751d4f thru 0x8759af8 (0x7daa bytes) 2024-09-22T01:01:36.111 INFO:teuthology.orchestra.run.smithi002.stdout:2409 write 0x419b891 thru 0x41a6115 (0xa885 bytes) 2024-09-22T01:01:36.118 INFO:teuthology.orchestra.run.smithi002.stdout:2410 writesame 0x5c5d5bb thru 0x5c64d87 data_size 0x1(0x77cd bytes) 2024-09-22T01:01:36.124 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:36.124 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:36.124 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:36.124 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:36.124 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:36.141 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:36.143 INFO:teuthology.orchestra.run.smithi002.stdout:2411 read 0x2dece38 thru 0x2dfaf82 (0xe14b bytes) 2024-09-22T01:01:36.146 INFO:teuthology.orchestra.run.smithi002.stdout:2413 trunc from 0x9416e75 to 0x84415bd 2024-09-22T01:01:36.158 INFO:teuthology.orchestra.run.smithi002.stdout:2414 read 0x6f2328 thru 0x6f8bd1 (0x68aa bytes) 2024-09-22T01:01:36.159 INFO:teuthology.orchestra.run.smithi002.stdout:2415 punch from 0x5c602 to 0x680a9, (0xbaa7 bytes) 2024-09-22T01:01:36.175 INFO:teuthology.orchestra.run.smithi002.stdout:2417 write 0xb6bf494 thru 0xb6c6914 (0x7481 bytes) 2024-09-22T01:01:36.182 INFO:teuthology.orchestra.run.smithi002.stdout:2418 write 0x9ec75f8 thru 0x9ed3a9a (0xc4a3 bytes) 2024-09-22T01:01:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:2419 trunc from 0xb6c6915 to 0x6eb47e6 2024-09-22T01:01:36.206 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:36.204+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:36.214 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:36.218 INFO:teuthology.orchestra.run.smithi002.stdout:2421 trunc from 0x6eb47e6 to 0x455527f 2024-09-22T01:01:36.245 INFO:teuthology.orchestra.run.smithi002.stdout:2422 write 0x4a8cd34 thru 0x4a923fa (0x56c7 bytes) 2024-09-22T01:01:36.251 INFO:teuthology.orchestra.run.smithi002.stdout:2423 read 0x463037d thru 0x46369f6 (0x667a bytes) 2024-09-22T01:01:36.251 INFO:teuthology.orchestra.run.smithi002.stdout:2424 write 0xdcdc8 thru 0xe6d73 (0x9fac bytes) 2024-09-22T01:01:36.255 INFO:teuthology.orchestra.run.smithi002.stdout:2426 read 0x34277d0 thru 0x3429987 (0x21b8 bytes) 2024-09-22T01:01:36.256 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:36.256 INFO:teuthology.orchestra.run.smithi002.stdout:2429 read 0x1b753f2 thru 0x1b82a42 (0xd651 bytes) 2024-09-22T01:01:36.257 INFO:teuthology.orchestra.run.smithi002.stdout:2431 write 0x450c869 thru 0x451672e (0x9ec6 bytes) 2024-09-22T01:01:36.262 INFO:teuthology.orchestra.run.smithi002.stdout:2432 read 0x301a0f0 thru 0x3029cd2 (0xfbe3 bytes) 2024-09-22T01:01:36.263 INFO:teuthology.orchestra.run.smithi002.stdout:2434 read 0x16d8ea0 thru 0x16db29e (0x23ff bytes) 2024-09-22T01:01:36.277 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32450 3.1b 3:dca5d1c0:::rbd_data.117cdc8a00a7.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.439609631, 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028862692999999998, 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32450 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837159+0000", 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837171+0000", 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2132e-05 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837199+0000", 2024-09-22T01:01:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7647000000000001e-05 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837211+0000", 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2231e-05 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837290+0000", 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8617999999999998e-05 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837416+0000", 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000126406 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.840665+0000", 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032488389999999999 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865675+0000", 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025009834000000002 2024-09-22T01:01:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865729+0000", 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4348999999999998e-05 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865890+0000", 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000161034 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.866019+0000", 2024-09-22T01:01:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012889600000000001 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32568 3.1b 3:dca5d1c0:::rbd_data.117cdc8a00a7.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.202936924999999, 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045259445000000002, 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32568 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073832+0000", 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.796e-06 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073844+0000", 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2276000000000001e-05 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073875+0000", 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1127999999999997e-05 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073894+0000", 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8352e-05 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.074008+0000", 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114034 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.074144+0000", 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013626600000000001 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.081862+0000", 2024-09-22T01:01:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077184369999999999 2024-09-22T01:01:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118757+0000", 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036894588999999998 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118852+0000", 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4911e-05 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118961+0000", 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109331 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.119088+0000", 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012732499999999999 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32757 3.1 3:822836a6:::rbd_data.117cdc8a00a7.000000000000002a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.709799446000002, 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040855028000000002, 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32757 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566969+0000", 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566980+0000", 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1742999999999999e-05 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567011+0000", 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0555000000000001e-05 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567025+0000", 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4046000000000001e-05 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567177+0000", 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015239100000000001 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567424+0000", 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024646200000000003 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.573230+0000", 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058063960000000001 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607542+0000", 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034311832 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607651+0000", 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108631 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607746+0000", 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4850000000000002e-05 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607822+0000", 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5943000000000001e-05 2024-09-22T01:01:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36334 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.520698423000001, 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.02825625, 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36334 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4000000000000002e-07 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756071+0000", 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9459999999999998e-06 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756080+0000", 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6430000000000001e-06 2024-09-22T01:01:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756086+0000", 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2489999999999998e-06 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.783797+0000", 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027711369999999999 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784023+0000", 2024-09-22T01:01:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022560200000000001 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784324+0000", 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000301 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36335 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.520671667, 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028810327, 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36335 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756095+0000", 2024-09-22T01:01:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756098+0000", 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8569999999999999e-06 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756106+0000", 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3070000000000003e-06 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756112+0000", 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784404+0000", 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028292048 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784621+0000", 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216865 2024-09-22T01:01:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784905+0000", 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028360199999999999 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36336 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.520645299, 2024-09-22T01:01:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029359435, 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36336 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756124+0000", 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9009999999999998e-06 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756132+0000", 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.259e-06 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756139+0000", 2024-09-22T01:01:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784985+0000", 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028846663000000002 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785202+0000", 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216648 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785480+0000", 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027814199999999999 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36338 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.520618857000001, 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029907710000000001, 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36338 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756148+0000", 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756150+0000", 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9399999999999998e-06 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756159+0000", 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3019999999999995e-06 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756165+0000", 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1879999999999999e-06 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785559+0000", 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029393586999999999 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785773+0000", 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021459499999999999 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.786055+0000", 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028168900000000001 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36509 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.507453668, 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025912054, 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36509 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769313+0000", 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0759999999999999e-06 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769331+0000", 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.753e-05 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769350+0000", 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9585e-05 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769365+0000", 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4926000000000001e-05 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.794791+0000", 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025425136000000001 2024-09-22T01:01:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.794998+0000", 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000207016 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795224+0000", 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022678499999999999 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36510 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.507381002000001, 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026348881000000001, 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36510 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769386+0000", 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9999999999999995e-07 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769393+0000", 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.153e-06 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769411+0000", 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8139e-05 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769425+0000", 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3579e-05 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795301+0000", 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025876399000000001 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795508+0000", 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020672600000000001 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795734+0000", 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225885 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36511 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.507321751999999, 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026956201999999999, 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36511 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769445+0000", 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0300000000000001e-06 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769451+0000", 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0270000000000001e-06 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769470+0000", 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769483+0000", 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3271999999999999e-05 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795809+0000", 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026326142 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796017+0000", 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020782600000000001 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796400+0000", 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038322600000000002 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36512 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.507263948, 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027282219, 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36512 2024-09-22T01:01:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769503+0000", 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769509+0000", 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999998e-06 2024-09-22T01:01:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769527+0000", 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8516e-05 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769541+0000", 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3586e-05 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796486+0000", 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026944941 2024-09-22T01:01:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796622+0000", 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013610200000000001 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796784+0000", 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000162468 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36513 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 638976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.507207028, 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027552297999999999, 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36513 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769560+0000", 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.06e-06 2024-09-22T01:01:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769565+0000", 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769583+0000", 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8122999999999999e-05 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769597+0000", 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3482000000000001e-05 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796834+0000", 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027237849000000001 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796956+0000", 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012197300000000001 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797111+0000", 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015474799999999999 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36514 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.507151780999999, 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027850182000000001, 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36514 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769615+0000", 2024-09-22T01:01:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769621+0000", 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6670000000000001e-06 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769639+0000", 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8389e-05 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769652+0000", 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2839e-05 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797204+0000", 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027551902 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797331+0000", 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012647 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797464+0000", 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133845 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36515 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.507094004000001, 2024-09-22T01:01:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028090786999999999, 2024-09-22T01:01:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36515 2024-09-22T01:01:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769673+0000", 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0100000000000001e-06 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769679+0000", 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8429999999999998e-06 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769698+0000", 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8759000000000001e-05 2024-09-22T01:01:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769711+0000", 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3699e-05 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797510+0000", 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027798442999999999 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797628+0000", 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118268 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797763+0000", 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013476499999999999 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36971 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 573440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.506023519999999, 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025899893, 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36971 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770744+0000", 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1030000000000001e-06 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770758+0000", 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4866e-05 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770777+0000", 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8782000000000001e-05 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770790+0000", 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3190000000000001e-05 2024-09-22T01:01:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.791603+0000", 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020812761999999999 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.791710+0000", 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010676200000000001 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796642+0000", 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004932428 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37890 3.1f 3:fada49d2:::rbd_data.117cd9a1164e.0000000000000026:12 [sparse-read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.138536165, 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026289204, 2024-09-22T01:01:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37890 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138231+0000", 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.92e-07 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138236+0000", 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.057e-06 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138249+0000", 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3794e-05 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138258+0000", 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5429999999999994e-06 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164056+0000", 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025798232000000001 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164220+0000", 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016423499999999999 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164519+0000", 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029855099999999998 2024-09-22T01:01:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37920 3.1b 3:d9ee1e9e:::rbd_data.117cf8aab9ed.0000000000000005:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.038928541000001, 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035889233999999999, 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37920 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237839+0000", 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.013e-06 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237850+0000", 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0473e-05 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237880+0000", 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0468e-05 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237902+0000", 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1569e-05 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.238081+0000", 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017904800000000001 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.238294+0000", 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021303500000000001 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.244279+0000", 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005984545 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273477+0000", 2024-09-22T01:01:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029198275999999999 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273560+0000", 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665000000000006e-05 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273661+0000", 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100839 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273727+0000", 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5302999999999995e-05 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38844 3.12 3:49c96a36:::rbd_data.117c55693b3.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7959881769999999, 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026176943000000001, 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38844 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480781+0000", 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.557e-06 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484901+0000", 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041200519999999999 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484932+0000", 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0261999999999999e-05 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484947+0000", 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4893e-05 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485088+0000", 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000141936 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485329+0000", 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024086600000000001 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485651+0000", 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032141 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.500650+0000", 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014999111000000001 2024-09-22T01:01:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506603+0000", 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059526689999999998 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506789+0000", 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186725 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506955+0000", 2024-09-22T01:01:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016546200000000001 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38846 3.1 3:823bb6bd:::rbd_data.117c55693b3.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7897509029999998, 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049688678999999999, 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38846 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487020+0000", 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.023e-06 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497105+0000", 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010085393 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497147+0000", 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1866999999999997e-05 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497210+0000", 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2324999999999997e-05 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497457+0000", 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024715099999999998 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497726+0000", 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026944000000000002 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.498317+0000", 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059106199999999995 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.516775+0000", 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018457952 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536527+0000", 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019751484 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536622+0000", 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5529e-05 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536704+0000", 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1452999999999994e-05 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:43461 3.1f 3:ff1f7135:::rbd_data.117c83c8bd76.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.596446+0000", 2024-09-22T01:01:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.680320013, 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040539668000000001, 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 43461 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596446+0000", 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596446+0000", 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596448+0000", 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3439999999999999e-06 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596473+0000", 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4402000000000001e-05 2024-09-22T01:01:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596501+0000", 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8072000000000002e-05 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596514+0000", 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596656+0000", 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014265799999999999 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596874+0000", 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218025 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.603496+0000", 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066221170000000003 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.636747+0000", 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033250698000000002 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.636827+0000", 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0296999999999994e-05 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.636925+0000", 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7872000000000001e-05 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.636986+0000", 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0340000000000002e-05 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.386 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:36.386 INFO:teuthology.orchestra.run.smithi002.stdout:2436 write 0xcb2cada thru 0xcb3480e (0x7d35 bytes) 2024-09-22T01:01:36.387 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:36.388 INFO:teuthology.orchestra.run.smithi002.stdout:2438 trunc from 0xcb3480f to 0x5d8a53f 2024-09-22T01:01:36.388 INFO:teuthology.orchestra.run.smithi002.stdout:2439 read 0x4645514 thru 0x465371f (0xe20c bytes) 2024-09-22T01:01:36.388 INFO:teuthology.orchestra.run.smithi002.stdout:2440 trunc from 0x5d8a53f to 0x801e5b 2024-09-22T01:01:36.388 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:36.384+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:36.407 INFO:teuthology.orchestra.run.smithi002.stdout:2441 write 0x65f0086 thru 0x65fad5c (0xacd7 bytes) 2024-09-22T01:01:36.431 INFO:teuthology.orchestra.run.smithi002.stdout:2443 write 0xaa5151b thru 0xaa5eead (0xd993 bytes) 2024-09-22T01:01:36.437 INFO:teuthology.orchestra.run.smithi002.stdout:2444 read 0x97ae4c9 thru 0x97bb854 (0xd38c bytes) 2024-09-22T01:01:36.450 INFO:teuthology.orchestra.run.smithi002.stdout:2445 write 0xb4b0cf8 thru 0xb4bcfa1 (0xc2aa bytes) 2024-09-22T01:01:36.471 INFO:teuthology.orchestra.run.smithi002.stdout:2446 writesame 0xc811d82 thru 0xc81f63a data_size 0x1(0xd8b9 bytes) 2024-09-22T01:01:36.471 INFO:teuthology.orchestra.run.smithi002.stdout:2447 read 0xbe7eae1 thru 0xbe83abc (0x4fdc bytes) 2024-09-22T01:01:36.471 INFO:teuthology.orchestra.run.smithi002.stdout:2449 read 0x26cdf30 thru 0x26d2646 (0x4717 bytes) 2024-09-22T01:01:36.471 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:36.471 INFO:teuthology.orchestra.run.smithi002.stdout:2451 writesame 0x1219d32 thru 0x121f0bf data_size 0xa(0x538e bytes) 2024-09-22T01:01:36.479 INFO:teuthology.orchestra.run.smithi002.stdout:2454 write 0x3354ca6 thru 0x3355c47 (0xfa2 bytes) 2024-09-22T01:01:36.486 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:36.487 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:36.483+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:36.510 INFO:teuthology.orchestra.run.smithi002.stdout:2457 trunc from 0xc81f63b to 0x16f7e04 2024-09-22T01:01:36.538 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:36.538 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:36.538 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:36.538 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:36.558 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:36.578 INFO:teuthology.orchestra.run.smithi002.stdout:2458 write 0xc8700d7 thru 0xc87958a (0x94b4 bytes) 2024-09-22T01:01:36.584 INFO:teuthology.orchestra.run.smithi002.stdout:2459 read 0x2ed9cf1 thru 0x2ee0b35 (0x6e45 bytes) 2024-09-22T01:01:36.584 INFO:teuthology.orchestra.run.smithi002.stdout:2460 writesame 0xae80cca thru 0xae817fe data_size 0x1(0xb35 bytes) 2024-09-22T01:01:36.587 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:36.583+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:36.592 INFO:teuthology.orchestra.run.smithi002.stdout:2461 write 0x4a02b89 thru 0x4a05987 (0x2dff bytes) 2024-09-22T01:01:36.615 INFO:teuthology.orchestra.run.smithi002.stdout:2462 write 0xeb8f94b thru 0xeb9df56 (0xe60c bytes) 2024-09-22T01:01:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:2463 trunc from 0xeb9df57 to 0x6f4817d 2024-09-22T01:01:36.659 INFO:teuthology.orchestra.run.smithi002.stdout:2465 punch from 0x1bdb40 to 0x1c943d, (0xb8fd bytes) 2024-09-22T01:01:36.660 INFO:teuthology.orchestra.run.smithi002.stdout:2466 read 0x9a8c6c thru 0x9af52d (0x68c2 bytes) 2024-09-22T01:01:36.681 INFO:teuthology.orchestra.run.smithi002.stdout:2467 writesame 0xaf7a65c thru 0xaf7bfab data_size 0x24(0x1950 bytes) 2024-09-22T01:01:36.681 INFO:teuthology.orchestra.run.smithi002.stdout:2468 writesame 0x44413bb thru 0x44481e5 data_size 0x15(0x6e2b bytes) 2024-09-22T01:01:36.688 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:36.688+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:36.689 INFO:teuthology.orchestra.run.smithi002.stdout:2469 writesame 0xfd8684 thru 0xfe239b data_size 0x2c(0x9d18 bytes) 2024-09-22T01:01:36.697 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:36.697 INFO:teuthology.orchestra.run.smithi002.stdout:2472 write 0xd447d thru 0xd5c87 (0x180b bytes) 2024-09-22T01:01:36.700 INFO:teuthology.orchestra.run.smithi002.stdout:2473 punch from 0x1763fe6 to 0x176e5c8, (0xa5e2 bytes) 2024-09-22T01:01:36.704 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:36.704 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:36.704 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:36.704 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:36.705 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:36.722 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:36.722 INFO:teuthology.orchestra.run.smithi002.stdout:2474 write 0xe158db3 thru 0xe16382e (0xaa7c bytes) 2024-09-22T01:01:36.728 INFO:teuthology.orchestra.run.smithi002.stdout:2475 read 0xc80f64b thru 0xc81b9a4 (0xc35a bytes) 2024-09-22T01:01:36.728 INFO:teuthology.orchestra.run.smithi002.stdout:2479 write 0x14771fc thru 0x147d42a (0x622f bytes) 2024-09-22T01:01:36.731 INFO:teuthology.orchestra.run.smithi002.stdout:2480 writesame 0xc1a2912 thru 0xc1ae5bd data_size 0xaf(0xbcac bytes) 2024-09-22T01:01:36.752 INFO:teuthology.orchestra.run.smithi002.stdout:2482 write 0xea80b99 thru 0xea82baf (0x2017 bytes) 2024-09-22T01:01:36.758 INFO:teuthology.orchestra.run.smithi002.stdout:2483 write 0x393a8de thru 0x3949b48 (0xf26b bytes) 2024-09-22T01:01:36.764 INFO:teuthology.orchestra.run.smithi002.stdout:2484 write 0xbc8cfb4 thru 0xbc9bf87 (0xefd4 bytes) 2024-09-22T01:01:36.770 INFO:teuthology.orchestra.run.smithi002.stdout:2485 read 0x55fa06a thru 0x5605628 (0xb5bf bytes) 2024-09-22T01:01:36.770 INFO:teuthology.orchestra.run.smithi002.stdout:2486 punch from 0x3ff9bee to 0x400744a, (0xd85c bytes) 2024-09-22T01:01:36.784 INFO:teuthology.orchestra.run.smithi002.stdout:2487 trunc from 0xea82bb0 to 0x6fab6f9 2024-09-22T01:01:36.789 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:36.783+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:36.821 INFO:teuthology.orchestra.run.smithi002.stdout:2488 trunc from 0x6fab6f9 to 0xc80418a 2024-09-22T01:01:36.835 INFO:teuthology.orchestra.run.smithi002.stdout:2489 read 0xeac94 thru 0xeaf6f (0x2dc bytes) 2024-09-22T01:01:36.836 INFO:teuthology.orchestra.run.smithi002.stdout:2491 write 0x6679dca thru 0x6687b56 (0xdd8d bytes) 2024-09-22T01:01:36.843 INFO:teuthology.orchestra.run.smithi002.stdout:2492 read 0x8f19ddb thru 0x8f287aa (0xe9d0 bytes) 2024-09-22T01:01:36.849 INFO:teuthology.orchestra.run.smithi002.stdout:2493 trunc from 0xc80418a to 0x98f79fa 2024-09-22T01:01:36.865 INFO:teuthology.orchestra.run.smithi002.stdout:2494 clone 24 order 23 su 524288 sc 13 2024-09-22T01:01:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786627156000002, 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786608297000001, 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786590111999999, 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786572323999998, 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786553765999997, 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786535714000003, 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786518364999999, 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786496411999998, 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786478877, 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786455091000001, 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786437229000001, 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786409992000003, 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786391287000001, 2024-09-22T01:01:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.78636779, 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786343821000003, 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.786324606000001, 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.785320056000003, 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.782866867999999, 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.782844787999998, 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.782814103, 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:36.958 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:36.959 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:36.959 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:36.955+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:37.051 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.7 to 1 (10000) 2024-09-22T01:01:37.058 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:01:37.059 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:37.056+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:37.078 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T01:01:37.078 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:01:37.143 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:37.143 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:37.160 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:37.161 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:37.155+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:37.194 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:37.244 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone23 from 0x98f79fa (overlap 0x801e5b) to 0x0 2024-09-22T01:01:37.285 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:37.283+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:37.342 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:37.342 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:37.342 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:37.342 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:37.342 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:37.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:37.362 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:37.360+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:37.368 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:37.368 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:37.388 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:37.388 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:37.406 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:37.406 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:37.422 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-22T01:01:37.424 INFO:teuthology.orchestra.run.smithi002.stdout:2495 trunc from 0x98f79fa to 0x911e996 2024-09-22T01:01:37.426 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:37.426 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:37.454 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:37.455 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:37.455 INFO:teuthology.orchestra.run.smithi002.stdout:2497 read 0x1159afe thru 0x1163079 (0x957c bytes) 2024-09-22T01:01:37.462 INFO:teuthology.orchestra.run.smithi002.stdout:2498 trunc from 0x911e996 to 0x3337769 2024-09-22T01:01:37.462 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:37.459+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:01:37.487 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:37.487 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:37.488 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:01:37.508 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:37.509 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:37.511 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-09-22T01:01:37.053919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":544574929,"num_objects":849,"num_object_clones":234,"num_object_copies":1698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":849,"num_whiteouts":380,"num_read":62834,"num_read_kb":265267,"num_write":6911,"num_write_kb":168719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":28205809,"num_keys_recovered":112,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5757,"ondisk_log_size":5757,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":424136,"kb_used_data":140160,"kb_used_omap":15,"kb_used_meta":283696,"kb_avail":754550584,"statfs":{"total":773094113280,"available":772659798016,"internally_reserved":0,"allocated":143523840,"data_stored":164439478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":290505401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78564949,"num_objects":215,"num_object_clones":54,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":145,"num_read":6988,"num_read_kb":26992,"num_write":1032,"num_write_kb":14412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.032772"},"pg_stats":[{"pgid":"3.1f","version":"60'179","reported_seq":887,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.636928+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:01:35.636928+0000","last_peered":"2024-09-22T01:01:35.636928+0000","last_clean":"2024-09-22T01:01:35.636928+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:01:35.636928+0000","last_undegraded":"2024-09-22T01:01:35.636928+0000","last_fullsized":"2024-09-22T01:01:35.636928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:21:22.869541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":27332772,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":816,"num_read_kb":9124,"num_write":219,"num_write_kb":6847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'311","reported_seq":1329,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.471857+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:01:35.471857+0000","last_peered":"2024-09-22T01:01:35.471857+0000","last_clean":"2024-09-22T01:01:35.471857+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:01:35.471857+0000","last_undegraded":"2024-09-22T01:01:35.471857+0000","last_fullsized":"2024-09-22T01:01:35.471857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:58.032979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":18299004,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":3059,"num_read_kb":8798,"num_write":415,"num_write_kb":5464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"60'163","reported_seq":2016,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.918999+0000","last_change":"2024-09-22T01:01:31.018985+0000","last_active":"2024-09-22T01:01:35.918999+0000","last_peered":"2024-09-22T01:01:35.918999+0000","last_clean":"2024-09-22T01:01:35.918999+0000","last_became_active":"2024-09-22T01:01:31.018372+0000","last_became_peered":"2024-09-22T01:01:31.018372+0000","last_unstale":"2024-09-22T01:01:35.918999+0000","last_undegraded":"2024-09-22T01:01:35.918999+0000","last_fullsized":"2024-09-22T01:01:35.918999+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:19:53.398523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17858619,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":2730,"num_read_kb":13319,"num_write":169,"num_write_kb":10004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":3845944,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"60'419","reported_seq":1976,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.650182+0000","last_change":"2024-09-22T01:01:31.022312+0000","last_active":"2024-09-22T01:01:35.650182+0000","last_peered":"2024-09-22T01:01:35.650182+0000","last_clean":"2024-09-22T01:01:35.650182+0000","last_became_active":"2024-09-22T01:01:31.019916+0000","last_became_peered":"2024-09-22T01:01:31.019916+0000","last_unstale":"2024-09-22T01:01:35.650182+0000","last_undegraded":"2024-09-22T01:01:35.650182+0000","last_fullsized":"2024-09-22T01:01:35.650182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:22.076367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33442140,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":3141,"num_read_kb":27790,"num_write":695,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4532656,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"60'100","reported_seq":308,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.331113+0000","last_change":"2024-09-22T01:00:49.730266+0000","last_active":"2024-09-22T01:01:35.331113+0000","last_peered":"2024-09-22T01:01:35.331113+0000","last_clean":"2024-09-22T01:01:35.331113+0000","last_became_active":"2024-09-22T01:00:47.730798+0000","last_became_peered":"2024-09-22T01:00:47.730798+0000","last_unstale":"2024-09-22T01:01:35.331113+0000","last_undegraded":"2024-09-22T01:01:35.331113+0000","last_fullsized":"2024-09-22T01:01:35.331113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:45:56.014813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":15964449,"num_objects":31,"num_object_clones":13,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":226,"num_read_kb":523,"num_write":95,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"58'98","reported_seq":1439,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.571108+0000","last_change":"2024-09-22T01:00:49.717588+0000","last_active":"2024-09-22T01:01:35.571108+0000","last_peered":"2024-09-22T01:01:35.571108+0000","last_clean":"2024-09-22T01:01:35.571108+0000","last_became_active":"2024-09-22T01:00:47.730348+0000","last_became_peered":"2024-09-22T01:00:47.730348+0000","last_unstale":"2024-09-22T01:01:35.571108+0000","last_undegraded":"2024-09-22T01:01:35.571108+0000","last_fullsized":"2024-09-22T01:01:35.571108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:38:36.521007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":20203244,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":1278,"num_read_kb":2105,"num_write":126,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"60'89","reported_seq":1776,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.196437+0000","last_change":"2024-09-22T01:00:49.708786+0000","last_active":"2024-09-22T01:01:35.196437+0000","last_peered":"2024-09-22T01:01:35.196437+0000","last_clean":"2024-09-22T01:01:35.196437+0000","last_became_active":"2024-09-22T01:00:47.723339+0000","last_became_peered":"2024-09-22T01:00:47.723339+0000","last_unstale":"2024-09-22T01:01:35.196437+0000","last_undegraded":"2024-09-22T01:01:35.196437+0000","last_fullsized":"2024-09-22T01:01:35.196437+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:46:27.696515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":14302259,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":1656,"num_read_kb":10783,"num_write":123,"num_write_kb":4568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"60'300","reported_seq":3022,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.473555+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:01:35.473555+0000","last_peered":"2024-09-22T01:01:35.473555+0000","last_clean":"2024-09-22T01:01:35.473555+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:01:35.473555+0000","last_undegraded":"2024-09-22T01:01:35.473555+0000","last_fullsized":"2024-09-22T01:01:35.473555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:54.153413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":34707789,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":3979,"num_read_kb":21708,"num_write":425,"num_write_kb":5994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"60'139","reported_seq":1422,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.641083+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:01:35.641083+0000","last_peered":"2024-09-22T01:01:35.641083+0000","last_clean":"2024-09-22T01:01:35.641083+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:01:35.641083+0000","last_undegraded":"2024-09-22T01:01:35.641083+0000","last_fullsized":"2024-09-22T01:01:35.641083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:38.432883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":5637978,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":11,"num_read":1610,"num_read_kb":4368,"num_write":150,"num_write_kb":1210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"60'100","reported_seq":142,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.918977+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:01:35.918977+0000","last_peered":"2024-09-22T01:01:35.918977+0000","last_clean":"2024-09-22T01:01:35.918977+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:01:35.918977+0000","last_undegraded":"2024-09-22T01:01:35.918977+0000","last_fullsized":"2024-09-22T01:01:35.918977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:34.519628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820655,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":1172,"num_read_kb":7270,"num_write":95,"num_write_kb":11364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"60'268","reported_seq":4467,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.481326+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:01:35.481326+0000","last_peered":"2024-09-22T01:01:35.481326+0000","last_clean":"2024-09-22T01:01:35.481326+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:01:35.481326+0000","last_undegraded":"2024-09-22T01:01:35.481326+0000","last_fullsized":"2024-09-22T01:01:35.481326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:54:24.143703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":21954502,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":5353,"num_read_kb":17703,"num_write":321,"num_write_kb":1175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"58'206","reported_seq":3010,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.231400+0000","last_change":"2024-09-22T01:00:49.716855+0000","last_active":"2024-09-22T01:01:35.231400+0000","last_peered":"2024-09-22T01:01:35.231400+0000","last_clean":"2024-09-22T01:01:35.231400+0000","last_became_active":"2024-09-22T01:00:47.735932+0000","last_became_peered":"2024-09-22T01:00:47.735932+0000","last_unstale":"2024-09-22T01:01:35.231400+0000","last_undegraded":"2024-09-22T01:01:35.231400+0000","last_fullsized":"2024-09-22T01:01:35.231400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:13.015741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":23355434,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":3555,"num_read_kb":9078,"num_write":248,"num_write_kb":2660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T01:01:36.066970+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:01:36.066970+0000","last_peered":"2024-09-22T01:01:36.066970+0000","last_clean":"2024-09-22T01:01:36.066970+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:01:36.066970+0000","last_undegraded":"2024-09-22T01:01:36.066970+0000","last_fullsized":"2024-09-22T01:01:36.066970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"60'150","reported_seq":1209,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.471800+0000","last_change":"2024-09-22T01:00:49.729709+0000","last_active":"2024-09-22T01:01:35.471800+0000","last_peered":"2024-09-22T01:01:35.471800+0000","last_clean":"2024-09-22T01:01:35.471800+0000","last_became_active":"2024-09-22T01:00:47.718964+0000","last_became_peered":"2024-09-22T01:00:47.718964+0000","last_unstale":"2024-09-22T01:01:35.471800+0000","last_undegraded":"2024-09-22T01:01:35.471800+0000","last_fullsized":"2024-09-22T01:01:35.471800+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:40.712430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":17659951,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":2219,"num_read_kb":3670,"num_write":135,"num_write_kb":1022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.031730+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:01:35.031730+0000","last_peered":"2024-09-22T01:01:35.031730+0000","last_clean":"2024-09-22T01:01:35.031730+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:01:35.031730+0000","last_undegraded":"2024-09-22T01:01:35.031730+0000","last_fullsized":"2024-09-22T01:01:35.031730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"60'302","reported_seq":1225,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.466668+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:01:35.466668+0000","last_peered":"2024-09-22T01:01:35.466668+0000","last_clean":"2024-09-22T01:01:35.466668+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:01:35.466668+0000","last_undegraded":"2024-09-22T01:01:35.466668+0000","last_fullsized":"2024-09-22T01:01:35.466668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:48:17.154355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":10997361,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1991,"num_read_kb":6209,"num_write":461,"num_write_kb":2215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080392+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:01:35.080392+0000","last_peered":"2024-09-22T01:01:35.080392+0000","last_clean":"2024-09-22T01:01:35.080392+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:01:35.080392+0000","last_undegraded":"2024-09-22T01:01:35.080392+0000","last_fullsized":"2024-09-22T01:01:35.080392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"60'141","reported_seq":2335,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.209114+0000","last_change":"2024-09-22T01:00:49.709328+0000","last_active":"2024-09-22T01:01:35.209114+0000","last_peered":"2024-09-22T01:01:35.209114+0000","last_clean":"2024-09-22T01:01:35.209114+0000","last_became_active":"2024-09-22T01:00:47.714621+0000","last_became_peered":"2024-09-22T01:00:47.714621+0000","last_unstale":"2024-09-22T01:01:35.209114+0000","last_undegraded":"2024-09-22T01:01:35.209114+0000","last_fullsized":"2024-09-22T01:01:35.209114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:09:55.732984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":9177117,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":2581,"num_read_kb":10846,"num_write":121,"num_write_kb":5547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.032321+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:01:35.032321+0000","last_peered":"2024-09-22T01:01:35.032321+0000","last_clean":"2024-09-22T01:01:35.032321+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:01:35.032321+0000","last_undegraded":"2024-09-22T01:01:35.032321+0000","last_fullsized":"2024-09-22T01:01:35.032321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"58'240","reported_seq":1772,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.052206+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:01:35.052206+0000","last_peered":"2024-09-22T01:01:35.052206+0000","last_clean":"2024-09-22T01:01:35.052206+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:01:35.052206+0000","last_undegraded":"2024-09-22T01:01:35.052206+0000","last_fullsized":"2024-09-22T01:01:35.052206+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:56:58.942501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8398136,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":3994,"num_read_kb":3731,"num_write":222,"num_write_kb":581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'173","reported_seq":638,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.037278+0000","last_change":"2024-09-22T01:00:49.716394+0000","last_active":"2024-09-22T01:01:35.037278+0000","last_peered":"2024-09-22T01:01:35.037278+0000","last_clean":"2024-09-22T01:01:35.037278+0000","last_became_active":"2024-09-22T01:00:47.721552+0000","last_became_peered":"2024-09-22T01:00:47.721552+0000","last_unstale":"2024-09-22T01:01:35.037278+0000","last_undegraded":"2024-09-22T01:01:35.037278+0000","last_fullsized":"2024-09-22T01:01:35.037278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:35.696905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":11492499,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":824,"num_read_kb":6095,"num_write":216,"num_write_kb":3775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.051948+0000","last_change":"2024-09-22T01:01:27.722404+0000","last_active":"2024-09-22T01:01:35.051948+0000","last_peered":"2024-09-22T01:01:35.051948+0000","last_clean":"2024-09-22T01:01:35.051948+0000","last_became_active":"2024-09-22T01:01:27.721384+0000","last_became_peered":"2024-09-22T01:01:27.721384+0000","last_unstale":"2024-09-22T01:01:35.051948+0000","last_undegraded":"2024-09-22T01:01:35.051948+0000","last_fullsized":"2024-09-22T01:01:35.051948+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:10.574832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"60'168","reported_seq":962,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.031391+0000","last_change":"2024-09-22T01:00:49.709391+0000","last_active":"2024-09-22T01:01:35.031391+0000","last_peered":"2024-09-22T01:01:35.031391+0000","last_clean":"2024-09-22T01:01:35.031391+0000","last_became_active":"2024-09-22T01:00:47.714544+0000","last_became_peered":"2024-09-22T01:00:47.714544+0000","last_unstale":"2024-09-22T01:01:35.031391+0000","last_undegraded":"2024-09-22T01:01:35.031391+0000","last_fullsized":"2024-09-22T01:01:35.031391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:07.858024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":19129394,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":1221,"num_read_kb":7079,"num_write":150,"num_write_kb":3610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.031254+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:01:35.031254+0000","last_peered":"2024-09-22T01:01:35.031254+0000","last_clean":"2024-09-22T01:01:35.031254+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:01:35.031254+0000","last_undegraded":"2024-09-22T01:01:35.031254+0000","last_fullsized":"2024-09-22T01:01:35.031254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"60'154","reported_seq":1441,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.121163+0000","last_change":"2024-09-22T01:00:49.730579+0000","last_active":"2024-09-22T01:01:35.121163+0000","last_peered":"2024-09-22T01:01:35.121163+0000","last_clean":"2024-09-22T01:01:35.121163+0000","last_became_active":"2024-09-22T01:00:47.734922+0000","last_became_peered":"2024-09-22T01:00:47.734922+0000","last_unstale":"2024-09-22T01:01:35.121163+0000","last_undegraded":"2024-09-22T01:01:35.121163+0000","last_fullsized":"2024-09-22T01:01:35.121163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:56.929087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":7680532,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":1505,"num_read_kb":6466,"num_write":183,"num_write_kb":7757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.079888+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:01:35.079888+0000","last_peered":"2024-09-22T01:01:35.079888+0000","last_clean":"2024-09-22T01:01:35.079888+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:01:35.079888+0000","last_undegraded":"2024-09-22T01:01:35.079888+0000","last_fullsized":"2024-09-22T01:01:35.079888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"58'143","reported_seq":1038,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.079332+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:01:35.079332+0000","last_peered":"2024-09-22T01:01:35.079332+0000","last_clean":"2024-09-22T01:01:35.079332+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:01:35.079332+0000","last_undegraded":"2024-09-22T01:01:35.079332+0000","last_fullsized":"2024-09-22T01:01:35.079332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:00:33.942064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":13099463,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":1023,"num_read_kb":5164,"num_write":166,"num_write_kb":11002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":51,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.051932+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:01:35.051932+0000","last_peered":"2024-09-22T01:01:35.051932+0000","last_clean":"2024-09-22T01:01:35.051932+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:01:35.051932+0000","last_undegraded":"2024-09-22T01:01:35.051932+0000","last_fullsized":"2024-09-22T01:01:35.051932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080170+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:01:35.080170+0000","last_peered":"2024-09-22T01:01:35.080170+0000","last_clean":"2024-09-22T01:01:35.080170+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:01:35.080170+0000","last_undegraded":"2024-09-22T01:01:35.080170+0000","last_fullsized":"2024-09-22T01:01:35.080170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"58'149","reported_seq":1736,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.079106+0000","last_change":"2024-09-22T01:00:49.735787+0000","last_active":"2024-09-22T01:01:35.079106+0000","last_peered":"2024-09-22T01:01:35.079106+0000","last_clean":"2024-09-22T01:01:35.079106+0000","last_became_active":"2024-09-22T01:00:47.720325+0000","last_became_peered":"2024-09-22T01:00:47.720325+0000","last_unstale":"2024-09-22T01:01:35.079106+0000","last_undegraded":"2024-09-22T01:01:35.079106+0000","last_fullsized":"2024-09-22T01:01:35.079106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:25.794935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":10243303,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2571,"num_read_kb":5411,"num_write":121,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"60'181","reported_seq":627,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.264880+0000","last_change":"2024-09-22T01:00:49.716175+0000","last_active":"2024-09-22T01:01:35.264880+0000","last_peered":"2024-09-22T01:01:35.264880+0000","last_clean":"2024-09-22T01:01:35.264880+0000","last_became_active":"2024-09-22T01:00:47.727151+0000","last_became_peered":"2024-09-22T01:00:47.727151+0000","last_unstale":"2024-09-22T01:01:35.264880+0000","last_undegraded":"2024-09-22T01:01:35.264880+0000","last_fullsized":"2024-09-22T01:01:35.264880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:16.756595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":22060119,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":976,"num_read_kb":1234,"num_write":244,"num_write_kb":3673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"61'241","reported_seq":1440,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T01:01:36.658816+0000","last_change":"2024-09-22T01:00:49.708641+0000","last_active":"2024-09-22T01:01:36.658816+0000","last_peered":"2024-09-22T01:01:36.658816+0000","last_clean":"2024-09-22T01:01:36.658816+0000","last_became_active":"2024-09-22T01:00:47.738845+0000","last_became_peered":"2024-09-22T01:00:47.738845+0000","last_unstale":"2024-09-22T01:01:36.658816+0000","last_undegraded":"2024-09-22T01:01:36.658816+0000","last_fullsized":"2024-09-22T01:01:36.658816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:59:54.869253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":13149040,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1963,"num_read_kb":2947,"num_write":288,"num_write_kb":3822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"60'208","reported_seq":1401,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.522972+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:01:35.522972+0000","last_peered":"2024-09-22T01:01:35.522972+0000","last_clean":"2024-09-22T01:01:35.522972+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:01:35.522972+0000","last_undegraded":"2024-09-22T01:01:35.522972+0000","last_fullsized":"2024-09-22T01:01:35.522972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:32:37.850499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":11625906,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":2591,"num_read_kb":13650,"num_write":236,"num_write_kb":6140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"60'121","reported_seq":269,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.931590+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:01:35.931590+0000","last_peered":"2024-09-22T01:01:35.931590+0000","last_clean":"2024-09-22T01:01:35.931590+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:01:35.931590+0000","last_undegraded":"2024-09-22T01:01:35.931590+0000","last_fullsized":"2024-09-22T01:01:35.931590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:33:31.692357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19244016,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":1917,"num_read_kb":9655,"num_write":113,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"60'80","reported_seq":1710,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.466869+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:01:35.466869+0000","last_peered":"2024-09-22T01:01:35.466869+0000","last_clean":"2024-09-22T01:01:35.466869+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:01:35.466869+0000","last_undegraded":"2024-09-22T01:01:35.466869+0000","last_fullsized":"2024-09-22T01:01:35.466869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:45:20.375033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":18681619,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":1486,"num_read_kb":17832,"num_write":81,"num_write_kb":12014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"58'172","reported_seq":1389,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080289+0000","last_change":"2024-09-22T01:00:49.863753+0000","last_active":"2024-09-22T01:01:35.080289+0000","last_peered":"2024-09-22T01:01:35.080289+0000","last_clean":"2024-09-22T01:01:35.080289+0000","last_became_active":"2024-09-22T01:00:47.732705+0000","last_became_peered":"2024-09-22T01:00:47.732705+0000","last_unstale":"2024-09-22T01:01:35.080289+0000","last_undegraded":"2024-09-22T01:01:35.080289+0000","last_fullsized":"2024-09-22T01:01:35.080289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:45:00.278161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":32716897,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":1366,"num_read_kb":9739,"num_write":178,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"58'100","reported_seq":776,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080179+0000","last_change":"2024-09-22T01:00:49.864401+0000","last_active":"2024-09-22T01:01:35.080179+0000","last_peered":"2024-09-22T01:01:35.080179+0000","last_clean":"2024-09-22T01:01:35.080179+0000","last_became_active":"2024-09-22T01:00:47.732593+0000","last_became_peered":"2024-09-22T01:00:47.732593+0000","last_unstale":"2024-09-22T01:01:35.080179+0000","last_undegraded":"2024-09-22T01:01:35.080179+0000","last_fullsized":"2024-09-22T01:01:35.080179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:38.738735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":9465374,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":1342,"num_read_kb":2111,"num_write":92,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"61'150","reported_seq":2073,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T01:01:36.527357+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:01:36.527357+0000","last_peered":"2024-09-22T01:01:36.527357+0000","last_clean":"2024-09-22T01:01:36.527357+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:01:36.527357+0000","last_undegraded":"2024-09-22T01:01:36.527357+0000","last_fullsized":"2024-09-22T01:01:36.527357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:17.454217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":25803863,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":2304,"num_read_kb":8893,"num_write":125,"num_write_kb":7442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"60'205","reported_seq":776,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080041+0000","last_change":"2024-09-22T01:00:49.863724+0000","last_active":"2024-09-22T01:01:35.080041+0000","last_peered":"2024-09-22T01:01:35.080041+0000","last_clean":"2024-09-22T01:01:35.080041+0000","last_became_active":"2024-09-22T01:00:47.733104+0000","last_became_peered":"2024-09-22T01:00:47.733104+0000","last_unstale":"2024-09-22T01:01:35.080041+0000","last_undegraded":"2024-09-22T01:01:35.080041+0000","last_fullsized":"2024-09-22T01:01:35.080041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:11.433225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":16775625,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":1211,"num_read_kb":3344,"num_write":299,"num_write_kb":1577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"59'88","reported_seq":443,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.039180+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:01:35.039180+0000","last_peered":"2024-09-22T01:01:35.039180+0000","last_clean":"2024-09-22T01:01:35.039180+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:01:35.039180+0000","last_undegraded":"2024-09-22T01:01:35.039180+0000","last_fullsized":"2024-09-22T01:01:35.039180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:52:59.112718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":9096816,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":285,"num_read_kb":705,"num_write":72,"num_write_kb":3401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"58'99","reported_seq":928,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080128+0000","last_change":"2024-09-22T01:00:49.864291+0000","last_active":"2024-09-22T01:01:35.080128+0000","last_peered":"2024-09-22T01:01:35.080128+0000","last_clean":"2024-09-22T01:01:35.080128+0000","last_became_active":"2024-09-22T01:00:47.733697+0000","last_became_peered":"2024-09-22T01:00:47.733697+0000","last_unstale":"2024-09-22T01:01:35.080128+0000","last_undegraded":"2024-09-22T01:01:35.080128+0000","last_fullsized":"2024-09-22T01:01:35.080128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:32:06.846099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":11608666,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":773,"num_read_kb":7817,"num_write":166,"num_write_kb":7387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":543984542,"num_objects":845,"num_object_clones":234,"num_object_copies":1690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":380,"num_read":62718,"num_read_kb":265167,"num_write":6750,"num_write_kb":166301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":28205809,"num_keys_recovered":112,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139227136,"data_stored":163168433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":5637,"ondisk_log_size":5637,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":6448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":6602752,"data_stored":7259307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299395,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57288,"kb_used_data":29176,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94314552,"statfs":{"total":96636764160,"available":96578101248,"internally_reserved":0,"allocated":29876224,"data_stored":36570983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57428,"kb_used_data":22076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94314412,"statfs":{"total":96636764160,"available":96577957888,"internally_reserved":0,"allocated":22605824,"data_stored":26578827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59392,"kb_used_data":24044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94312448,"statfs":{"total":96636764160,"available":96575946752,"internally_reserved":0,"allocated":24621056,"data_stored":27911220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66716,"kb_used_data":22944,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94305124,"statfs":{"total":96636764160,"available":96568446976,"internally_reserved":0,"allocated":23494656,"data_stored":24980540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":4800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":4915200,"data_stored":6698457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42488,"kb_used_data":7140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94329352,"statfs":{"total":96636764160,"available":96593256448,"internally_reserved":0,"allocated":7311360,"data_stored":9227342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63036,"kb_used_data":23532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94308804,"statfs":{"total":96636764160,"available":96572215296,"internally_reserved":0,"allocated":24096768,"data_stored":25212802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22986752,"data_stored":24469958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6619136,"data_stored":8918666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4235264,"data_stored":6395642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22966272,"data_stored":24813309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23953408,"data_stored":27621063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29184000,"data_stored":36262317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5906432,"data_stored":6947189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23375872,"data_stored":27740289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:01:37.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:01:37.514 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:37.514 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:37.524 INFO:teuthology.orchestra.run.smithi002.stdout:2500 write 0x909559f thru 0x909f92e (0xa390 bytes) 2024-09-22T01:01:37.529 INFO:teuthology.orchestra.run.smithi002.stdout:2503 punch from 0x81d8700 to 0x81d994d, (0x124d bytes) 2024-09-22T01:01:37.533 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:37.536 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:37.536 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:37.541 INFO:teuthology.orchestra.run.smithi002.stdout:2504 write 0xd022a0d thru 0xd0329e0 (0xffd4 bytes) 2024-09-22T01:01:37.545 INFO:teuthology.orchestra.run.smithi002.stdout:2505 write 0x4069ea5 thru 0x4079ca7 (0xfe03 bytes) 2024-09-22T01:01:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:2506 trunc from 0xd0329e1 to 0x7a72303 2024-09-22T01:01:37.564 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:37.564+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:37.570 INFO:teuthology.orchestra.run.smithi002.stdout:2507 punch from 0x744d8e2 to 0x744f0ac, (0x17ca bytes) 2024-09-22T01:01:37.570 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:37.570 INFO:teuthology.orchestra.run.smithi002.stdout:2509 punch from 0x36f1338 to 0x36fd158, (0xbe20 bytes) 2024-09-22T01:01:37.570 INFO:teuthology.orchestra.run.smithi002.stdout:2511 read 0x2915278 thru 0x2922083 (0xce0c bytes) 2024-09-22T01:01:37.580 INFO:teuthology.orchestra.run.smithi002.stdout:2512 write 0xac4a01f thru 0xac502df (0x62c1 bytes) 2024-09-22T01:01:37.585 INFO:teuthology.orchestra.run.smithi002.stdout:2513 read 0x1713e28 thru 0x1718b65 (0x4d3e bytes) 2024-09-22T01:01:37.586 INFO:teuthology.orchestra.run.smithi002.stdout:2514 clone 25 order 18 su 2048 sc 14 2024-09-22T01:01:37.654 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:37.654 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:37.665 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:37.664+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:37.671 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:37.765 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:37.764+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:37.825 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:37.825 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:37.829 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:01:37.829 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:01:37.840 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:37.848 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-09-22T01:01:37.053919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":544574929,"num_objects":849,"num_object_clones":234,"num_object_copies":1698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":849,"num_whiteouts":380,"num_read":62834,"num_read_kb":265267,"num_write":6911,"num_write_kb":168719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":28205809,"num_keys_recovered":112,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5757,"ondisk_log_size":5757,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":424136,"kb_used_data":140160,"kb_used_omap":15,"kb_used_meta":283696,"kb_avail":754550584,"statfs":{"total":773094113280,"available":772659798016,"internally_reserved":0,"allocated":143523840,"data_stored":164439478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":290505401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78564949,"num_objects":215,"num_object_clones":54,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":145,"num_read":6988,"num_read_kb":26992,"num_write":1032,"num_write_kb":14412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.032772"},"pg_stats":[{"pgid":"3.1f","version":"60'179","reported_seq":887,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.636928+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:01:35.636928+0000","last_peered":"2024-09-22T01:01:35.636928+0000","last_clean":"2024-09-22T01:01:35.636928+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:01:35.636928+0000","last_undegraded":"2024-09-22T01:01:35.636928+0000","last_fullsized":"2024-09-22T01:01:35.636928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:21:22.869541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":27332772,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":816,"num_read_kb":9124,"num_write":219,"num_write_kb":6847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'311","reported_seq":1329,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.471857+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:01:35.471857+0000","last_peered":"2024-09-22T01:01:35.471857+0000","last_clean":"2024-09-22T01:01:35.471857+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:01:35.471857+0000","last_undegraded":"2024-09-22T01:01:35.471857+0000","last_fullsized":"2024-09-22T01:01:35.471857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:58.032979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":18299004,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":3059,"num_read_kb":8798,"num_write":415,"num_write_kb":5464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"60'163","reported_seq":2016,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.918999+0000","last_change":"2024-09-22T01:01:31.018985+0000","last_active":"2024-09-22T01:01:35.918999+0000","last_peered":"2024-09-22T01:01:35.918999+0000","last_clean":"2024-09-22T01:01:35.918999+0000","last_became_active":"2024-09-22T01:01:31.018372+0000","last_became_peered":"2024-09-22T01:01:31.018372+0000","last_unstale":"2024-09-22T01:01:35.918999+0000","last_undegraded":"2024-09-22T01:01:35.918999+0000","last_fullsized":"2024-09-22T01:01:35.918999+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:19:53.398523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17858619,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":2730,"num_read_kb":13319,"num_write":169,"num_write_kb":10004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":3845944,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"60'419","reported_seq":1976,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.650182+0000","last_change":"2024-09-22T01:01:31.022312+0000","last_active":"2024-09-22T01:01:35.650182+0000","last_peered":"2024-09-22T01:01:35.650182+0000","last_clean":"2024-09-22T01:01:35.650182+0000","last_became_active":"2024-09-22T01:01:31.019916+0000","last_became_peered":"2024-09-22T01:01:31.019916+0000","last_unstale":"2024-09-22T01:01:35.650182+0000","last_undegraded":"2024-09-22T01:01:35.650182+0000","last_fullsized":"2024-09-22T01:01:35.650182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:22.076367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33442140,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":3141,"num_read_kb":27790,"num_write":695,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4532656,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"60'100","reported_seq":308,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.331113+0000","last_change":"2024-09-22T01:00:49.730266+0000","last_active":"2024-09-22T01:01:35.331113+0000","last_peered":"2024-09-22T01:01:35.331113+0000","last_clean":"2024-09-22T01:01:35.331113+0000","last_became_active":"2024-09-22T01:00:47.730798+0000","last_became_peered":"2024-09-22T01:00:47.730798+0000","last_unstale":"2024-09-22T01:01:35.331113+0000","last_undegraded":"2024-09-22T01:01:35.331113+0000","last_fullsized":"2024-09-22T01:01:35.331113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:45:56.014813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":15964449,"num_objects":31,"num_object_clones":13,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":226,"num_read_kb":523,"num_write":95,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"58'98","reported_seq":1439,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.571108+0000","last_change":"2024-09-22T01:00:49.717588+0000","last_active":"2024-09-22T01:01:35.571108+0000","last_peered":"2024-09-22T01:01:35.571108+0000","last_clean":"2024-09-22T01:01:35.571108+0000","last_became_active":"2024-09-22T01:00:47.730348+0000","last_became_peered":"2024-09-22T01:00:47.730348+0000","last_unstale":"2024-09-22T01:01:35.571108+0000","last_undegraded":"2024-09-22T01:01:35.571108+0000","last_fullsized":"2024-09-22T01:01:35.571108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:38:36.521007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":20203244,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":1278,"num_read_kb":2105,"num_write":126,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"60'89","reported_seq":1776,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.196437+0000","last_change":"2024-09-22T01:00:49.708786+0000","last_active":"2024-09-22T01:01:35.196437+0000","last_peered":"2024-09-22T01:01:35.196437+0000","last_clean":"2024-09-22T01:01:35.196437+0000","last_became_active":"2024-09-22T01:00:47.723339+0000","last_became_peered":"2024-09-22T01:00:47.723339+0000","last_unstale":"2024-09-22T01:01:35.196437+0000","last_undegraded":"2024-09-22T01:01:35.196437+0000","last_fullsized":"2024-09-22T01:01:35.196437+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:46:27.696515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":14302259,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":1656,"num_read_kb":10783,"num_write":123,"num_write_kb":4568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"60'300","reported_seq":3022,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.473555+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:01:35.473555+0000","last_peered":"2024-09-22T01:01:35.473555+0000","last_clean":"2024-09-22T01:01:35.473555+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:01:35.473555+0000","last_undegraded":"2024-09-22T01:01:35.473555+0000","last_fullsized":"2024-09-22T01:01:35.473555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:54.153413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":34707789,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":3979,"num_read_kb":21708,"num_write":425,"num_write_kb":5994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"60'139","reported_seq":1422,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.641083+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:01:35.641083+0000","last_peered":"2024-09-22T01:01:35.641083+0000","last_clean":"2024-09-22T01:01:35.641083+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:01:35.641083+0000","last_undegraded":"2024-09-22T01:01:35.641083+0000","last_fullsized":"2024-09-22T01:01:35.641083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:38.432883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":5637978,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":11,"num_read":1610,"num_read_kb":4368,"num_write":150,"num_write_kb":1210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"60'100","reported_seq":142,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.918977+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:01:35.918977+0000","last_peered":"2024-09-22T01:01:35.918977+0000","last_clean":"2024-09-22T01:01:35.918977+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:01:35.918977+0000","last_undegraded":"2024-09-22T01:01:35.918977+0000","last_fullsized":"2024-09-22T01:01:35.918977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:34.519628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820655,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":1172,"num_read_kb":7270,"num_write":95,"num_write_kb":11364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"60'268","reported_seq":4467,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.481326+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:01:35.481326+0000","last_peered":"2024-09-22T01:01:35.481326+0000","last_clean":"2024-09-22T01:01:35.481326+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:01:35.481326+0000","last_undegraded":"2024-09-22T01:01:35.481326+0000","last_fullsized":"2024-09-22T01:01:35.481326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:54:24.143703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":21954502,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":5353,"num_read_kb":17703,"num_write":321,"num_write_kb":1175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"58'206","reported_seq":3010,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.231400+0000","last_change":"2024-09-22T01:00:49.716855+0000","last_active":"2024-09-22T01:01:35.231400+0000","last_peered":"2024-09-22T01:01:35.231400+0000","last_clean":"2024-09-22T01:01:35.231400+0000","last_became_active":"2024-09-22T01:00:47.735932+0000","last_became_peered":"2024-09-22T01:00:47.735932+0000","last_unstale":"2024-09-22T01:01:35.231400+0000","last_undegraded":"2024-09-22T01:01:35.231400+0000","last_fullsized":"2024-09-22T01:01:35.231400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:13.015741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":23355434,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":3555,"num_read_kb":9078,"num_write":248,"num_write_kb":2660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T01:01:36.066970+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:01:36.066970+0000","last_peered":"2024-09-22T01:01:36.066970+0000","last_clean":"2024-09-22T01:01:36.066970+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:01:36.066970+0000","last_undegraded":"2024-09-22T01:01:36.066970+0000","last_fullsized":"2024-09-22T01:01:36.066970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"60'150","reported_seq":1209,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.471800+0000","last_change":"2024-09-22T01:00:49.729709+0000","last_active":"2024-09-22T01:01:35.471800+0000","last_peered":"2024-09-22T01:01:35.471800+0000","last_clean":"2024-09-22T01:01:35.471800+0000","last_became_active":"2024-09-22T01:00:47.718964+0000","last_became_peered":"2024-09-22T01:00:47.718964+0000","last_unstale":"2024-09-22T01:01:35.471800+0000","last_undegraded":"2024-09-22T01:01:35.471800+0000","last_fullsized":"2024-09-22T01:01:35.471800+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:40.712430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":17659951,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":2219,"num_read_kb":3670,"num_write":135,"num_write_kb":1022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.031730+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:01:35.031730+0000","last_peered":"2024-09-22T01:01:35.031730+0000","last_clean":"2024-09-22T01:01:35.031730+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:01:35.031730+0000","last_undegraded":"2024-09-22T01:01:35.031730+0000","last_fullsized":"2024-09-22T01:01:35.031730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"60'302","reported_seq":1225,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.466668+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:01:35.466668+0000","last_peered":"2024-09-22T01:01:35.466668+0000","last_clean":"2024-09-22T01:01:35.466668+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:01:35.466668+0000","last_undegraded":"2024-09-22T01:01:35.466668+0000","last_fullsized":"2024-09-22T01:01:35.466668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:48:17.154355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":10997361,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1991,"num_read_kb":6209,"num_write":461,"num_write_kb":2215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080392+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:01:35.080392+0000","last_peered":"2024-09-22T01:01:35.080392+0000","last_clean":"2024-09-22T01:01:35.080392+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:01:35.080392+0000","last_undegraded":"2024-09-22T01:01:35.080392+0000","last_fullsized":"2024-09-22T01:01:35.080392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"60'141","reported_seq":2335,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.209114+0000","last_change":"2024-09-22T01:00:49.709328+0000","last_active":"2024-09-22T01:01:35.209114+0000","last_peered":"2024-09-22T01:01:35.209114+0000","last_clean":"2024-09-22T01:01:35.209114+0000","last_became_active":"2024-09-22T01:00:47.714621+0000","last_became_peered":"2024-09-22T01:00:47.714621+0000","last_unstale":"2024-09-22T01:01:35.209114+0000","last_undegraded":"2024-09-22T01:01:35.209114+0000","last_fullsized":"2024-09-22T01:01:35.209114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:09:55.732984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":9177117,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":2581,"num_read_kb":10846,"num_write":121,"num_write_kb":5547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.032321+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:01:35.032321+0000","last_peered":"2024-09-22T01:01:35.032321+0000","last_clean":"2024-09-22T01:01:35.032321+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:01:35.032321+0000","last_undegraded":"2024-09-22T01:01:35.032321+0000","last_fullsized":"2024-09-22T01:01:35.032321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"58'240","reported_seq":1772,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.052206+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:01:35.052206+0000","last_peered":"2024-09-22T01:01:35.052206+0000","last_clean":"2024-09-22T01:01:35.052206+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:01:35.052206+0000","last_undegraded":"2024-09-22T01:01:35.052206+0000","last_fullsized":"2024-09-22T01:01:35.052206+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:56:58.942501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8398136,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":3994,"num_read_kb":3731,"num_write":222,"num_write_kb":581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'173","reported_seq":638,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.037278+0000","last_change":"2024-09-22T01:00:49.716394+0000","last_active":"2024-09-22T01:01:35.037278+0000","last_peered":"2024-09-22T01:01:35.037278+0000","last_clean":"2024-09-22T01:01:35.037278+0000","last_became_active":"2024-09-22T01:00:47.721552+0000","last_became_peered":"2024-09-22T01:00:47.721552+0000","last_unstale":"2024-09-22T01:01:35.037278+0000","last_undegraded":"2024-09-22T01:01:35.037278+0000","last_fullsized":"2024-09-22T01:01:35.037278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:35.696905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":11492499,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":824,"num_read_kb":6095,"num_write":216,"num_write_kb":3775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.051948+0000","last_change":"2024-09-22T01:01:27.722404+0000","last_active":"2024-09-22T01:01:35.051948+0000","last_peered":"2024-09-22T01:01:35.051948+0000","last_clean":"2024-09-22T01:01:35.051948+0000","last_became_active":"2024-09-22T01:01:27.721384+0000","last_became_peered":"2024-09-22T01:01:27.721384+0000","last_unstale":"2024-09-22T01:01:35.051948+0000","last_undegraded":"2024-09-22T01:01:35.051948+0000","last_fullsized":"2024-09-22T01:01:35.051948+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:10.574832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"60'168","reported_seq":962,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.031391+0000","last_change":"2024-09-22T01:00:49.709391+0000","last_active":"2024-09-22T01:01:35.031391+0000","last_peered":"2024-09-22T01:01:35.031391+0000","last_clean":"2024-09-22T01:01:35.031391+0000","last_became_active":"2024-09-22T01:00:47.714544+0000","last_became_peered":"2024-09-22T01:00:47.714544+0000","last_unstale":"2024-09-22T01:01:35.031391+0000","last_undegraded":"2024-09-22T01:01:35.031391+0000","last_fullsized":"2024-09-22T01:01:35.031391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:07.858024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":19129394,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":1221,"num_read_kb":7079,"num_write":150,"num_write_kb":3610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.031254+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:01:35.031254+0000","last_peered":"2024-09-22T01:01:35.031254+0000","last_clean":"2024-09-22T01:01:35.031254+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:01:35.031254+0000","last_undegraded":"2024-09-22T01:01:35.031254+0000","last_fullsized":"2024-09-22T01:01:35.031254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"60'154","reported_seq":1441,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.121163+0000","last_change":"2024-09-22T01:00:49.730579+0000","last_active":"2024-09-22T01:01:35.121163+0000","last_peered":"2024-09-22T01:01:35.121163+0000","last_clean":"2024-09-22T01:01:35.121163+0000","last_became_active":"2024-09-22T01:00:47.734922+0000","last_became_peered":"2024-09-22T01:00:47.734922+0000","last_unstale":"2024-09-22T01:01:35.121163+0000","last_undegraded":"2024-09-22T01:01:35.121163+0000","last_fullsized":"2024-09-22T01:01:35.121163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:56.929087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":7680532,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":1505,"num_read_kb":6466,"num_write":183,"num_write_kb":7757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.079888+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:01:35.079888+0000","last_peered":"2024-09-22T01:01:35.079888+0000","last_clean":"2024-09-22T01:01:35.079888+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:01:35.079888+0000","last_undegraded":"2024-09-22T01:01:35.079888+0000","last_fullsized":"2024-09-22T01:01:35.079888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"58'143","reported_seq":1038,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.079332+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:01:35.079332+0000","last_peered":"2024-09-22T01:01:35.079332+0000","last_clean":"2024-09-22T01:01:35.079332+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:01:35.079332+0000","last_undegraded":"2024-09-22T01:01:35.079332+0000","last_fullsized":"2024-09-22T01:01:35.079332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:00:33.942064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":13099463,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":1023,"num_read_kb":5164,"num_write":166,"num_write_kb":11002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":51,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.051932+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:01:35.051932+0000","last_peered":"2024-09-22T01:01:35.051932+0000","last_clean":"2024-09-22T01:01:35.051932+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:01:35.051932+0000","last_undegraded":"2024-09-22T01:01:35.051932+0000","last_fullsized":"2024-09-22T01:01:35.051932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080170+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:01:35.080170+0000","last_peered":"2024-09-22T01:01:35.080170+0000","last_clean":"2024-09-22T01:01:35.080170+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:01:35.080170+0000","last_undegraded":"2024-09-22T01:01:35.080170+0000","last_fullsized":"2024-09-22T01:01:35.080170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"58'149","reported_seq":1736,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.079106+0000","last_change":"2024-09-22T01:00:49.735787+0000","last_active":"2024-09-22T01:01:35.079106+0000","last_peered":"2024-09-22T01:01:35.079106+0000","last_clean":"2024-09-22T01:01:35.079106+0000","last_became_active":"2024-09-22T01:00:47.720325+0000","last_became_peered":"2024-09-22T01:00:47.720325+0000","last_unstale":"2024-09-22T01:01:35.079106+0000","last_undegraded":"2024-09-22T01:01:35.079106+0000","last_fullsized":"2024-09-22T01:01:35.079106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:25.794935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":10243303,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2571,"num_read_kb":5411,"num_write":121,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"60'181","reported_seq":627,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.264880+0000","last_change":"2024-09-22T01:00:49.716175+0000","last_active":"2024-09-22T01:01:35.264880+0000","last_peered":"2024-09-22T01:01:35.264880+0000","last_clean":"2024-09-22T01:01:35.264880+0000","last_became_active":"2024-09-22T01:00:47.727151+0000","last_became_peered":"2024-09-22T01:00:47.727151+0000","last_unstale":"2024-09-22T01:01:35.264880+0000","last_undegraded":"2024-09-22T01:01:35.264880+0000","last_fullsized":"2024-09-22T01:01:35.264880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:16.756595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":22060119,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":976,"num_read_kb":1234,"num_write":244,"num_write_kb":3673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"61'241","reported_seq":1440,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T01:01:36.658816+0000","last_change":"2024-09-22T01:00:49.708641+0000","last_active":"2024-09-22T01:01:36.658816+0000","last_peered":"2024-09-22T01:01:36.658816+0000","last_clean":"2024-09-22T01:01:36.658816+0000","last_became_active":"2024-09-22T01:00:47.738845+0000","last_became_peered":"2024-09-22T01:00:47.738845+0000","last_unstale":"2024-09-22T01:01:36.658816+0000","last_undegraded":"2024-09-22T01:01:36.658816+0000","last_fullsized":"2024-09-22T01:01:36.658816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:59:54.869253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":13149040,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1963,"num_read_kb":2947,"num_write":288,"num_write_kb":3822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"60'208","reported_seq":1401,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.522972+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:01:35.522972+0000","last_peered":"2024-09-22T01:01:35.522972+0000","last_clean":"2024-09-22T01:01:35.522972+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:01:35.522972+0000","last_undegraded":"2024-09-22T01:01:35.522972+0000","last_fullsized":"2024-09-22T01:01:35.522972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:32:37.850499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":11625906,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":2591,"num_read_kb":13650,"num_write":236,"num_write_kb":6140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"60'121","reported_seq":269,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.931590+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:01:35.931590+0000","last_peered":"2024-09-22T01:01:35.931590+0000","last_clean":"2024-09-22T01:01:35.931590+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:01:35.931590+0000","last_undegraded":"2024-09-22T01:01:35.931590+0000","last_fullsized":"2024-09-22T01:01:35.931590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:33:31.692357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19244016,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":1917,"num_read_kb":9655,"num_write":113,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"60'80","reported_seq":1710,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.466869+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:01:35.466869+0000","last_peered":"2024-09-22T01:01:35.466869+0000","last_clean":"2024-09-22T01:01:35.466869+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:01:35.466869+0000","last_undegraded":"2024-09-22T01:01:35.466869+0000","last_fullsized":"2024-09-22T01:01:35.466869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:45:20.375033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":18681619,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":1486,"num_read_kb":17832,"num_write":81,"num_write_kb":12014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"58'172","reported_seq":1389,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080289+0000","last_change":"2024-09-22T01:00:49.863753+0000","last_active":"2024-09-22T01:01:35.080289+0000","last_peered":"2024-09-22T01:01:35.080289+0000","last_clean":"2024-09-22T01:01:35.080289+0000","last_became_active":"2024-09-22T01:00:47.732705+0000","last_became_peered":"2024-09-22T01:00:47.732705+0000","last_unstale":"2024-09-22T01:01:35.080289+0000","last_undegraded":"2024-09-22T01:01:35.080289+0000","last_fullsized":"2024-09-22T01:01:35.080289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:45:00.278161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":32716897,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":1366,"num_read_kb":9739,"num_write":178,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"58'100","reported_seq":776,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080179+0000","last_change":"2024-09-22T01:00:49.864401+0000","last_active":"2024-09-22T01:01:35.080179+0000","last_peered":"2024-09-22T01:01:35.080179+0000","last_clean":"2024-09-22T01:01:35.080179+0000","last_became_active":"2024-09-22T01:00:47.732593+0000","last_became_peered":"2024-09-22T01:00:47.732593+0000","last_unstale":"2024-09-22T01:01:35.080179+0000","last_undegraded":"2024-09-22T01:01:35.080179+0000","last_fullsized":"2024-09-22T01:01:35.080179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:38.738735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":9465374,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":1342,"num_read_kb":2111,"num_write":92,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"61'150","reported_seq":2073,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T01:01:36.527357+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:01:36.527357+0000","last_peered":"2024-09-22T01:01:36.527357+0000","last_clean":"2024-09-22T01:01:36.527357+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:01:36.527357+0000","last_undegraded":"2024-09-22T01:01:36.527357+0000","last_fullsized":"2024-09-22T01:01:36.527357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:17.454217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":25803863,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":2304,"num_read_kb":8893,"num_write":125,"num_write_kb":7442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"60'205","reported_seq":776,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080041+0000","last_change":"2024-09-22T01:00:49.863724+0000","last_active":"2024-09-22T01:01:35.080041+0000","last_peered":"2024-09-22T01:01:35.080041+0000","last_clean":"2024-09-22T01:01:35.080041+0000","last_became_active":"2024-09-22T01:00:47.733104+0000","last_became_peered":"2024-09-22T01:00:47.733104+0000","last_unstale":"2024-09-22T01:01:35.080041+0000","last_undegraded":"2024-09-22T01:01:35.080041+0000","last_fullsized":"2024-09-22T01:01:35.080041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:11.433225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":16775625,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":1211,"num_read_kb":3344,"num_write":299,"num_write_kb":1577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"59'88","reported_seq":443,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.039180+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:01:35.039180+0000","last_peered":"2024-09-22T01:01:35.039180+0000","last_clean":"2024-09-22T01:01:35.039180+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:01:35.039180+0000","last_undegraded":"2024-09-22T01:01:35.039180+0000","last_fullsized":"2024-09-22T01:01:35.039180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:52:59.112718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":9096816,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":285,"num_read_kb":705,"num_write":72,"num_write_kb":3401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"58'99","reported_seq":928,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T01:01:35.080128+0000","last_change":"2024-09-22T01:00:49.864291+0000","last_active":"2024-09-22T01:01:35.080128+0000","last_peered":"2024-09-22T01:01:35.080128+0000","last_clean":"2024-09-22T01:01:35.080128+0000","last_became_active":"2024-09-22T01:00:47.733697+0000","last_became_peered":"2024-09-22T01:00:47.733697+0000","last_unstale":"2024-09-22T01:01:35.080128+0000","last_undegraded":"2024-09-22T01:01:35.080128+0000","last_fullsized":"2024-09-22T01:01:35.080128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:32:06.846099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":11608666,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":773,"num_read_kb":7817,"num_write":166,"num_write_kb":7387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":543984542,"num_objects":845,"num_object_clones":234,"num_object_copies":1690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":380,"num_read":62718,"num_read_kb":265167,"num_write":6750,"num_write_kb":166301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":28205809,"num_keys_recovered":112,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139227136,"data_stored":163168433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":5637,"ondisk_log_size":5637,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":6448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":6602752,"data_stored":7259307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299395,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57288,"kb_used_data":29176,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94314552,"statfs":{"total":96636764160,"available":96578101248,"internally_reserved":0,"allocated":29876224,"data_stored":36570983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57428,"kb_used_data":22076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94314412,"statfs":{"total":96636764160,"available":96577957888,"internally_reserved":0,"allocated":22605824,"data_stored":26578827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59392,"kb_used_data":24044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94312448,"statfs":{"total":96636764160,"available":96575946752,"internally_reserved":0,"allocated":24621056,"data_stored":27911220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66716,"kb_used_data":22944,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94305124,"statfs":{"total":96636764160,"available":96568446976,"internally_reserved":0,"allocated":23494656,"data_stored":24980540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":4800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":4915200,"data_stored":6698457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42488,"kb_used_data":7140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94329352,"statfs":{"total":96636764160,"available":96593256448,"internally_reserved":0,"allocated":7311360,"data_stored":9227342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63036,"kb_used_data":23532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94308804,"statfs":{"total":96636764160,"available":96572215296,"internally_reserved":0,"allocated":24096768,"data_stored":25212802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22986752,"data_stored":24469958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6619136,"data_stored":8918666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4235264,"data_stored":6395642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22966272,"data_stored":24813309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23953408,"data_stored":27621063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29184000,"data_stored":36262317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5906432,"data_stored":6947189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23375872,"data_stored":27740289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:01:37.850 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T01:01:37.851 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:01:37.851 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:01:37.851 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-22T01:01:37.889 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:37.887+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:37.959 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:37.959 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:37.967 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:37.963+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:37.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:38.065 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:01:38.067 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:38.064+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:38.106 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:38.106 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:38.122 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:38.208 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:38.208+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:38.212 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-22T01:01:38.207+0000 7fa2a81e5640 -1 mon.a@0(leader).osd e63 definitely_dead 0 2024-09-22T01:01:38.245 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone24 from 0xac502e0 (overlap 0x3337769) to 0x1ebae0b 2024-09-22T01:01:38.268 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:38.263+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:38.288 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:38.288 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:38.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:38.369 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:38.363+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:38.466 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:38.467 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:38.470 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:38.467+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:38.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:38.539 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-22T01:01:38.542 INFO:teuthology.orchestra.run.smithi002.stdout:2520 trunc from 0xac502e0 to 0xa78592f 2024-09-22T01:01:38.572 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:38.568+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:38.619 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:38.619 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:38.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:38.665 INFO:teuthology.orchestra.run.smithi002.stdout:2521 read 0x88150 thru 0x96519 (0xe3ca bytes) 2024-09-22T01:01:38.672 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:38.667+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:38.697 INFO:teuthology.orchestra.run.smithi002.stdout:2522 trunc from 0xa78592f to 0x47d7844 2024-09-22T01:01:38.777 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:38.767+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:38.824 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:38.825 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:38.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:38.844 INFO:teuthology.orchestra.run.smithi002.stdout:2524 writesame 0x4346a99 thru 0x434bd2c data_size 0x46(0x5294 bytes) 2024-09-22T01:01:38.884 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:38.884+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:38.951 INFO:teuthology.orchestra.run.smithi002.stdout:2525 trunc from 0x47d7844 to 0xe9e0f0e 2024-09-22T01:01:38.962 INFO:teuthology.orchestra.run.smithi002.stdout:2527 punch from 0xdcb6be8 to 0xdcc6289, (0xf6a1 bytes) 2024-09-22T01:01:38.962 INFO:teuthology.orchestra.run.smithi002.stdout:2528 write 0xa821932 thru 0xa82c40c (0xaadb bytes) 2024-09-22T01:01:38.974 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:38.971+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:38.978 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:38.978 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:38.980 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:38.980 INFO:teuthology.orchestra.run.smithi002.stdout:2531 write 0x3b03f6a thru 0x3b09155 (0x51ec bytes) 2024-09-22T01:01:38.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:39.059 INFO:teuthology.orchestra.run.smithi002.stdout:2532 read 0x6fc6b6f thru 0x6fcbdc4 (0x5256 bytes) 2024-09-22T01:01:39.059 INFO:teuthology.orchestra.run.smithi002.stdout:2535 read 0xd2a1815 thru 0xd2aa343 (0x8b2f bytes) 2024-09-22T01:01:39.059 INFO:teuthology.orchestra.run.smithi002.stdout:2536 write 0x16002ba thru 0x160fab9 (0xf800 bytes) 2024-09-22T01:01:39.073 INFO:teuthology.orchestra.run.smithi002.stderr:marked out osd.4. 2024-09-22T01:01:39.076 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:39.071+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:39.175 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:39.175 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:39.177 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:39.176+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:39.188 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:39.278 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:39.275+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:39.352 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:39.352 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:39.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:39.412 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:39.412+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:39.479 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:39.476+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:39.489 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:39.489 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:39.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:39.567 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:39.614 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:39.611+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:39.666 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:39.666 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:39.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:01:39.681 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:39.675+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:39.726 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:39.726 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:39.743 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:39.743 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:39.760 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:39.761 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:39.777 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:39.777 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:39.781 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:39.780+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:39.797 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:39.798 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:39.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:01:39.882 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:39.879+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:39.976 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:39.976 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:39.983 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:39.979+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:39.990 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:40.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:01:40.089 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:40.084+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:40.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:40.117 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:40.130 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:40.130 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:40.130 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:40.130 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:40.139 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:40.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:40.165 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:40.165 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:40.185 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:40.184+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:40.286 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:40.284+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:40.287 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:40.287 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:40.302 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:40.387 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:40.383+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:40.423 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:40.423 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:40.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:40.424 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:40.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.424 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32450 3.1b 3:dca5d1c0:::rbd_data.117cdc8a00a7.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:40.424 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:40.424 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.585075083, 2024-09-22T01:01:40.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028862692999999998, 2024-09-22T01:01:40.424 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32450 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837156+0000", 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837159+0000", 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-22T01:01:40.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837171+0000", 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2132e-05 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837199+0000", 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7647000000000001e-05 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837211+0000", 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2231e-05 2024-09-22T01:01:40.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837290+0000", 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8617999999999998e-05 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.837416+0000", 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000126406 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.840665+0000", 2024-09-22T01:01:40.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032488389999999999 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865675+0000", 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025009834000000002 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865729+0000", 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4348999999999998e-05 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:40.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.865890+0000", 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000161034 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:17.866019+0000", 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012889600000000001 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32568 3.1b 3:dca5d1c0:::rbd_data.117cdc8a00a7.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:40.430 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.348402376999999, 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045259445000000002, 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32568 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073829+0000", 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073832+0000", 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.796e-06 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073844+0000", 2024-09-22T01:01:40.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2276000000000001e-05 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073875+0000", 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1127999999999997e-05 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.073894+0000", 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8352e-05 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.074008+0000", 2024-09-22T01:01:40.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114034 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.074144+0000", 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013626600000000001 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.081862+0000", 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077184369999999999 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118757+0000", 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036894588999999998 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118852+0000", 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4911e-05 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.118961+0000", 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109331 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.119088+0000", 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012732499999999999 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:32757 3.1 3:822836a6:::rbd_data.117cdc8a00a7.000000000000002a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.855264898000001, 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040855028000000002, 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 32757 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566967+0000", 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566969+0000", 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.566980+0000", 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1742999999999999e-05 2024-09-22T01:01:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567011+0000", 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0555000000000001e-05 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567025+0000", 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4046000000000001e-05 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567177+0000", 2024-09-22T01:01:40.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015239100000000001 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.567424+0000", 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024646200000000003 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.573230+0000", 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058063960000000001 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:40.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607542+0000", 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034311832 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607651+0000", 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108631 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607746+0000", 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4850000000000002e-05 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.607822+0000", 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5943000000000001e-05 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36334 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.666163874999999, 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.02825625, 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36334 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756068+0000", 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4000000000000002e-07 2024-09-22T01:01:40.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756071+0000", 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9459999999999998e-06 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756080+0000", 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6430000000000001e-06 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756086+0000", 2024-09-22T01:01:40.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2489999999999998e-06 2024-09-22T01:01:40.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.783797+0000", 2024-09-22T01:01:40.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027711369999999999 2024-09-22T01:01:40.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784023+0000", 2024-09-22T01:01:40.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022560200000000001 2024-09-22T01:01:40.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784324+0000", 2024-09-22T01:01:40.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000301 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36335 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.666137118999998, 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028810327, 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36335 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756094+0000", 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756095+0000", 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756098+0000", 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8569999999999999e-06 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756106+0000", 2024-09-22T01:01:40.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3070000000000003e-06 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756112+0000", 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784404+0000", 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028292048 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784621+0000", 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216865 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784905+0000", 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028360199999999999 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36336 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.666110751000001, 2024-09-22T01:01:40.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029359435, 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36336 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756121+0000", 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756124+0000", 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9009999999999998e-06 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756132+0000", 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.259e-06 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756139+0000", 2024-09-22T01:01:40.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.784985+0000", 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028846663000000002 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785202+0000", 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216648 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785480+0000", 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027814199999999999 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36338 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.666084308999999, 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029907710000000001, 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36338 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756147+0000", 2024-09-22T01:01:40.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756148+0000", 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756150+0000", 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9399999999999998e-06 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756159+0000", 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3019999999999995e-06 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.756165+0000", 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1879999999999999e-06 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785559+0000", 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029393586999999999 2024-09-22T01:01:40.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.785773+0000", 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021459499999999999 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.786055+0000", 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028168900000000001 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36509 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.65291912, 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025912054, 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36509 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769312+0000", 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769313+0000", 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0759999999999999e-06 2024-09-22T01:01:40.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769331+0000", 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.753e-05 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769350+0000", 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9585e-05 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769365+0000", 2024-09-22T01:01:40.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4926000000000001e-05 2024-09-22T01:01:40.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.794791+0000", 2024-09-22T01:01:40.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025425136000000001 2024-09-22T01:01:40.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.794998+0000", 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000207016 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795224+0000", 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022678499999999999 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36510 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.652846453999999, 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026348881000000001, 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36510 2024-09-22T01:01:40.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.468 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769385+0000", 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769386+0000", 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9999999999999995e-07 2024-09-22T01:01:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769393+0000", 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.153e-06 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769411+0000", 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8139e-05 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769425+0000", 2024-09-22T01:01:40.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3579e-05 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795301+0000", 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025876399000000001 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795508+0000", 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020672600000000001 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795734+0000", 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225885 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36511 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.652787203999999, 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026956201999999999, 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36511 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769444+0000", 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769445+0000", 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0300000000000001e-06 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769451+0000", 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0270000000000001e-06 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769470+0000", 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769483+0000", 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3271999999999999e-05 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.795809+0000", 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026326142 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796017+0000", 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020782600000000001 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796400+0000", 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038322600000000002 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.476 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36512 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.652729399999998, 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027282219, 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36512 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769502+0000", 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769503+0000", 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769509+0000", 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999998e-06 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769527+0000", 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8516e-05 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769541+0000", 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3586e-05 2024-09-22T01:01:40.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796486+0000", 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026944941 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796622+0000", 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013610200000000001 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796784+0000", 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000162468 2024-09-22T01:01:40.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36513 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 638976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.65267248, 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027552297999999999, 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36513 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769559+0000", 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769560+0000", 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.06e-06 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769565+0000", 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769583+0000", 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8122999999999999e-05 2024-09-22T01:01:40.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769597+0000", 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3482000000000001e-05 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796834+0000", 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027237849000000001 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796956+0000", 2024-09-22T01:01:40.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012197300000000001 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797111+0000", 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015474799999999999 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36514 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.652617233000001, 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027850182000000001, 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36514 2024-09-22T01:01:40.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769614+0000", 2024-09-22T01:01:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769615+0000", 2024-09-22T01:01:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:01:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769621+0000", 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6670000000000001e-06 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769639+0000", 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8389e-05 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769652+0000", 2024-09-22T01:01:40.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2839e-05 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797204+0000", 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027551902 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797331+0000", 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012647 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797464+0000", 2024-09-22T01:01:40.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133845 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36515 3.6 3:675c54e9:::rbd_data.117cdc8a00a7.0000000000000021:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.652559455999999, 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028090786999999999, 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36515 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769672+0000", 2024-09-22T01:01:40.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769673+0000", 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0100000000000001e-06 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769679+0000", 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8429999999999998e-06 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769698+0000", 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8759000000000001e-05 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.769711+0000", 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3699e-05 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797510+0000", 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027798442999999999 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797628+0000", 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118268 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.797763+0000", 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013476499999999999 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:36971 3.17 3:ecaf1103:::rbd_data.117cdc8a00a7.0000000000000007:10 [read 573440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.651488971999999, 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025899893, 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36971 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770742+0000", 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770744+0000", 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1030000000000001e-06 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770758+0000", 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4866e-05 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770777+0000", 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8782000000000001e-05 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.770790+0000", 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3190000000000001e-05 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.791603+0000", 2024-09-22T01:01:40.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020812761999999999 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.791710+0000", 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010676200000000001 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.796642+0000", 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004932428 2024-09-22T01:01:40.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37890 3.1f 3:fada49d2:::rbd_data.117cd9a1164e.0000000000000026:12 [sparse-read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.284001617000001, 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026289204, 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37890 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138230+0000", 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138231+0000", 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.92e-07 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138236+0000", 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.057e-06 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138249+0000", 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3794e-05 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.138258+0000", 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5429999999999994e-06 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164056+0000", 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025798232000000001 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164220+0000", 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016423499999999999 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.164519+0000", 2024-09-22T01:01:40.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029855099999999998 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:37920 3.1b 3:d9ee1e9e:::rbd_data.117cf8aab9ed.0000000000000005:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.184393993, 2024-09-22T01:01:40.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035889233999999999, 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37920 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237837+0000", 2024-09-22T01:01:40.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237839+0000", 2024-09-22T01:01:40.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.013e-06 2024-09-22T01:01:40.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237850+0000", 2024-09-22T01:01:40.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0473e-05 2024-09-22T01:01:40.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237880+0000", 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0468e-05 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.237902+0000", 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1569e-05 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.238081+0000", 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017904800000000001 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.238294+0000", 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021303500000000001 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.244279+0000", 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005984545 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273477+0000", 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029198275999999999 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273560+0000", 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665000000000006e-05 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273661+0000", 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100839 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.273727+0000", 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5302999999999995e-05 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.510 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38844 3.12 3:49c96a36:::rbd_data.117c55693b3.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.941453629, 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026176943000000001, 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38844 2024-09-22T01:01:40.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480778+0000", 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.480781+0000", 2024-09-22T01:01:40.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.557e-06 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484901+0000", 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041200519999999999 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484932+0000", 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0261999999999999e-05 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.484947+0000", 2024-09-22T01:01:40.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4893e-05 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485088+0000", 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000141936 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485329+0000", 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024086600000000001 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:40.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.485651+0000", 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032141 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.500650+0000", 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014999111000000001 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506603+0000", 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059526689999999998 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:40.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506789+0000", 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186725 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.506955+0000", 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016546200000000001 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38846 3.1 3:823bb6bd:::rbd_data.117c55693b3.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.935216355, 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049688678999999999, 2024-09-22T01:01:40.516 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38846 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487020+0000", 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.023e-06 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497105+0000", 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010085393 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497147+0000", 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1866999999999997e-05 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497210+0000", 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2324999999999997e-05 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497457+0000", 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024715099999999998 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497726+0000", 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026944000000000002 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.498317+0000", 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059106199999999995 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.516775+0000", 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018457952 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536527+0000", 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019751484 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536622+0000", 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5529e-05 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536704+0000", 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1452999999999994e-05 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.521 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.522 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:43461 3.1f 3:ff1f7135:::rbd_data.117c83c8bd76.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-22T01:01:40.522 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.596446+0000", 2024-09-22T01:01:40.522 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8257854650000001, 2024-09-22T01:01:40.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040539668000000001, 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 43461 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596446+0000", 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596446+0000", 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596448+0000", 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3439999999999999e-06 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596473+0000", 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4402000000000001e-05 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596501+0000", 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8072000000000002e-05 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596514+0000", 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596656+0000", 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014265799999999999 2024-09-22T01:01:40.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.596874+0000", 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218025 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.603496+0000", 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066221170000000003 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:40.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.636747+0000", 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033250698000000002 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.636827+0000", 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0296999999999994e-05 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.636925+0000", 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7872000000000001e-05 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:40.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.636986+0000", 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0340000000000002e-05 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:40.529 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:40.531 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:40.531 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:40.531+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:40.632 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:40.627+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:40.707 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:40.707 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:40.707 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:40.707 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:40.721 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:40.733 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:40.732+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:40.833 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:40.832+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:40.894 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:40.894 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:40.894 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:40.894 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:40.895 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:40.908 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:40.954 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:40.951+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:41.035 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:41.032+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957670082, 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.042 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957651222999999, 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:41.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:41.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:41.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957633037999997, 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:41.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957615250000003, 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:41.051 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:41.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957596692000003, 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:41.060 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957578640000001, 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:41.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957561290999998, 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:41.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:41.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957539337999997, 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:41.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957521802999999, 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957498016999999, 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:41.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957480154999999, 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:41.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:41.087 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957452918000001, 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:41.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:41.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:41.091 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957434212999999, 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:41.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957410715999998, 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957386747000001, 2024-09-22T01:01:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.957367531999999, 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:41.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.956362982000002, 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:41.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:41.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:41.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.953909793999998, 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:41.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.953887713999997, 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:41.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:41.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.953857028999998, 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:41.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.126 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:41.127 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:41.128 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:01:41.174 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:41.171+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:41.237 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:41.236+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:41.243 INFO:teuthology.orchestra.run.smithi002.stdout:2537 write 0x7084c4 thru 0x710d3e (0x887b bytes) 2024-09-22T01:01:41.338 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:41.336+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:41.357 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:41.358 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:41.376 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:41.393 INFO:teuthology.orchestra.run.smithi002.stdout:2538 read 0xb801801 thru 0xb808e9a (0x769a bytes) 2024-09-22T01:01:41.397 INFO:teuthology.orchestra.run.smithi002.stdout:2539 trunc from 0xe9e0f0e to 0xb3544c7 2024-09-22T01:01:41.404 INFO:teuthology.orchestra.run.smithi002.stdout:2540 read 0x5d21bf3 thru 0x5d2ab09 (0x8f17 bytes) 2024-09-22T01:01:41.404 INFO:teuthology.orchestra.run.smithi002.stdout:2542 read 0x1f7173 thru 0x1fc8f0 (0x577e bytes) 2024-09-22T01:01:41.407 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:41.420 INFO:teuthology.orchestra.run.smithi002.stdout:2544 trunc from 0xb3544c7 to 0x2706b8 2024-09-22T01:01:41.439 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:41.436+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:41.539 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:41.536+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:41.555 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:41.555 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:41.574 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:41.606 INFO:teuthology.orchestra.run.smithi002.stdout:2546 write 0x2b433b4 thru 0x2b4e68d (0xb2da bytes) 2024-09-22T01:01:41.622 INFO:teuthology.orchestra.run.smithi002.stdout:2547 write 0x1de089e thru 0x1de5d49 (0x54ac bytes) 2024-09-22T01:01:41.636 INFO:teuthology.orchestra.run.smithi002.stdout:2548 punch from 0xfae75e to 0xfb3923, (0x51c5 bytes) 2024-09-22T01:01:41.636 INFO:teuthology.orchestra.run.smithi002.stdout:2550 read 0xcbc10 thru 0xdb3b5 (0xf7a6 bytes) 2024-09-22T01:01:41.640 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:41.640+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:41.645 INFO:teuthology.orchestra.run.smithi002.stdout:2551 write 0x36036e1 thru 0x360fbb9 (0xc4d9 bytes) 2024-09-22T01:01:41.664 INFO:teuthology.orchestra.run.smithi002.stdout:2552 writesame 0x218870a thru 0x2191c10 data_size 0x51(0x9507 bytes) 2024-09-22T01:01:41.683 INFO:teuthology.orchestra.run.smithi002.stdout:2553 writesame 0x109f176 thru 0x10ada16 data_size 0x27(0xe8a1 bytes) 2024-09-22T01:01:41.719 INFO:teuthology.orchestra.run.smithi002.stdout:2555 write 0xe3172ce thru 0xe32510e (0xde41 bytes) 2024-09-22T01:01:41.736 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:41.736 INFO:teuthology.orchestra.run.smithi002.stdout:2557 write 0xdae9cc thru 0xdb552f (0x6b64 bytes) 2024-09-22T01:01:41.741 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:41.740+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:41.751 INFO:teuthology.orchestra.run.smithi002.stdout:2559 read 0x37d3587 thru 0x37d7318 (0x3d92 bytes) 2024-09-22T01:01:41.752 INFO:teuthology.orchestra.run.smithi002.stdout:2561 punch from 0x7edbb6a to 0x7ee8c56, (0xd0ec bytes) 2024-09-22T01:01:41.752 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:41.752 INFO:teuthology.orchestra.run.smithi002.stdout:2563 punch from 0x71e130b to 0x71eacf4, (0x99e9 bytes) 2024-09-22T01:01:41.753 INFO:teuthology.orchestra.run.smithi002.stdout:2564 write 0x470ffc6 thru 0x4719447 (0x9482 bytes) 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8964 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'20, mlcod=32'17)", 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.620801694999997, 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077894110000000004, 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136525+0000", 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3540000000000001e-06 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137650+0000", 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011257190000000001 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137671+0000", 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0560999999999998e-05 2024-09-22T01:01:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137680+0000", 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137763+0000", 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2472999999999995e-05 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137793+0000", 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-05 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144267+0000", 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064739139999999999 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144313+0000", 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5624999999999998e-05 2024-09-22T01:01:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8968 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'21, mlcod=32'17)", 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.616375650000002, 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004894771, 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140952+0000", 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1849999999999998e-06 2024-09-22T01:01:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.141985+0000", 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001032984 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142001+0000", 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6776999999999999e-05 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142010+0000", 2024-09-22T01:01:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4640000000000006e-06 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142101+0000", 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1312999999999998e-05 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142119+0000", 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8202e-05 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145810+0000", 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036909970000000001 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145844+0000", 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3849000000000001e-05 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9077 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'31, mlcod=32'29)", 2024-09-22T01:01:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.362051274999999, 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034796760000000001, 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395277+0000", 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8600000000000001e-06 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395294+0000", 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7513e-05 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395337+0000", 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2749e-05 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395358+0000", 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1097000000000001e-05 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395548+0000", 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189829 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395593+0000", 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4919000000000001e-05 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398634+0000", 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003041373 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398753+0000", 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119336 2024-09-22T01:01:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9083 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'32, mlcod=32'30)", 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.356269994000002, 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002882763, 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401055+0000", 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401057+0000", 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.677e-06 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401069+0000", 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2673e-05 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401107+0000", 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7200000000000003e-05 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401122+0000", 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5203e-05 2024-09-22T01:01:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401251+0000", 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012935900000000001 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.401287+0000", 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5633000000000003e-05 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.403853+0000", 2024-09-22T01:01:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025658389999999999 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.403938+0000", 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5179000000000006e-05 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:32673 3.0 e45/22 3:05ac80b1:::rbd_data.117cdc8a00a7.0000000000000002:head v 45'136, mlcod=45'134)", 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.380844+0000", 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.376481098999999, 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027056850000000002, 2024-09-22T01:01:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.380844+0000", 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.380844+0000", 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.380846+0000", 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.380860+0000", 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4132e-05 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.380889+0000", 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8529000000000001e-05 2024-09-22T01:01:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.380908+0000", 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8859e-05 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.381248+0000", 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033986999999999998 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.381318+0000", 2024-09-22T01:01:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0572000000000001e-05 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.383406+0000", 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020878210000000001 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.383550+0000", 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014366499999999999 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:32773 3.0 e45/22 3:05ac80b1:::rbd_data.117cdc8a00a7.0000000000000002:head v 45'137, mlcod=45'135)", 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.640160+0000", 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.117165106000002, 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00276553, 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.640160+0000", 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.640160+0000", 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.640162+0000", 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.559e-06 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.640181+0000", 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8145999999999999e-05 2024-09-22T01:01:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.640224+0000", 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3408000000000002e-05 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.640247+0000", 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2804999999999999e-05 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.640460+0000", 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021330700000000001 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.640521+0000", 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1342999999999999e-05 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.642806+0000", 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022848830000000001 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.642925+0000", 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119079 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:32822 3.0 e45/22 3:076669d9:::rbd_data.117cdc8a00a7.0000000000000022:head v 45'138, mlcod=45'136)", 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:18.732422+0000", 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.024902790999999, 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002379384, 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.732422+0000", 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.732422+0000", 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.732425+0000", 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6539999999999999e-06 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.732470+0000", 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5133000000000003e-05 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.732519+0000", 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8767000000000003e-05 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.732544+0000", 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5392999999999999e-05 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.732719+0000", 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017488900000000001 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.732782+0000", 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2637000000000004e-05 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.734679+0000", 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018970020000000001 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:18.734802+0000", 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012290900000000001 2024-09-22T01:01:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35894 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'145, mlcod=47'143)", 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.604310397999999, 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055540160000000002, 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8700000000000002e-07 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153852+0000", 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083735000000000005 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153865+0000", 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2804e-05 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153872+0000", 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153943+0000", 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0908000000000006e-05 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153968+0000", 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5530000000000001e-05 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158535+0000", 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045670559999999999 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158569+0000", 2024-09-22T01:01:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210999999999998e-05 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35895 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'147, mlcod=47'143)", 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.601510242, 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031074570000000001, 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155816+0000", 2024-09-22T01:01:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.666e-06 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155839+0000", 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2768e-05 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155940+0000", 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010056999999999999 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155982+0000", 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1850000000000001e-05 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.156671+0000", 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00068920600000000004 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.156712+0000", 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0917999999999997e-05 2024-09-22T01:01:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158885+0000", 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021735859999999999 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158922+0000", 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6893000000000001e-05 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37899 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'73, mlcod=48'71)", 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.577806575, 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012095786000000001, 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179519+0000", 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4099999999999997e-07 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183241+0000", 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037216110000000001 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183261+0000", 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9799000000000001e-05 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183274+0000", 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3046e-05 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183374+0000", 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010009399999999999 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183399+0000", 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5375999999999999e-05 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191540+0000", 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081403029999999998 2024-09-22T01:01:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191614+0000", 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4715999999999994e-05 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37936 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'74, mlcod=48'72)", 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.408045504, 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003111749, 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349282+0000", 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2970000000000002e-06 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349299+0000", 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7278999999999998e-05 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349331+0000", 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2218000000000002e-05 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349346+0000", 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4856e-05 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349548+0000", 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020148599999999999 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349608+0000", 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9893999999999999e-05 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.352260+0000", 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002652583 2024-09-22T01:01:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.352391+0000", 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013113600000000001 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:39987 3.9 e56/22 3:91104e29:::rbd_data.117c35b6a0e9.000000000000022f:head v 56'87, mlcod=56'85)", 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:31.394923+0000", 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.362402302, 2024-09-22T01:01:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028934690000000001, 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.394923+0000", 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.394923+0000", 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.394924+0000", 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5710000000000001e-06 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.394935+0000", 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0264e-05 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.394961+0000", 2024-09-22T01:01:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6528000000000001e-05 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.394972+0000", 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0543000000000001e-05 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.395105+0000", 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000133454 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.395166+0000", 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1414000000000005e-05 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.397595+0000", 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002428099 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:31.397816+0000", 2024-09-22T01:01:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022159600000000001 2024-09-22T01:01:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:43527 3.9 e60/22 3:96fe8936:::rbd_data.117c83c8bd76.000000000000001b:head v 60'182, mlcod=60'180)", 2024-09-22T01:01:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9157553639999998, 2024-09-22T01:01:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005051725, 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841572+0000", 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845187+0000", 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036148880000000001 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845210+0000", 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2926e-05 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845221+0000", 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.17e-05 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845322+0000", 2024-09-22T01:01:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101003 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845348+0000", 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5361e-05 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846545+0000", 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011971659999999999 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846621+0000", 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6631000000000005e-05 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:43533 3.9 e60/22 3:9605ef45:::rbd_data.117c83c8bd76.0000000000000013:head v 60'184, mlcod=60'182)", 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.917030+0000", 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8402946, 2024-09-22T01:01:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023719499999999998, 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917030+0000", 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917030+0000", 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917033+0000", 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917051+0000", 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8569000000000001e-05 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917093+0000", 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1912000000000001e-05 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917114+0000", 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0893000000000001e-05 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917281+0000", 2024-09-22T01:01:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166766 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917337+0000", 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5705000000000003e-05 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.919233+0000", 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001896565 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.919402+0000", 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016912399999999999 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:43537 3.9 e60/22 3:96fe8936:::rbd_data.117c83c8bd76.000000000000001b:head v 60'185, mlcod=60'182)", 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.917254+0000", 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8400712229999998, 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026502539999999999, 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917254+0000", 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917254+0000", 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917255+0000", 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.274e-06 2024-09-22T01:01:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917266+0000", 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1273e-05 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917290+0000", 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3411999999999999e-05 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.917305+0000", 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5702999999999999e-05 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.918093+0000", 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078771699999999995 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.918145+0000", 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1802999999999999e-05 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.919805+0000", 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001660176 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.919904+0000", 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8895999999999996e-05 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(3.13 66/65 [PushOp(3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f, version: 44'54, data_included: [65536~65536,393216~131072,589824~131072,786432~65536,1048576~65536,1179648~65536,1507328~131072,1662436~172572,1900544~65536,2097152~131072,2293760~24036], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f@44'54, size: 7732453, copy_subset: [(0, 7732453)], clone_subset: {}, snapset: f=[]:{f: [f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2317796, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.416116+0000", 2024-09-22T01:01:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34120859399999998, 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027315479999999999, 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.416116+0000", 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.416116+0000", 2024-09-22T01:01:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.416118+0000", 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9259999999999999e-06 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.417300+0000", 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001181278 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.417341+0000", 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1081000000000002e-05 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.417368+0000", 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7673000000000001e-05 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.417506+0000", 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013740099999999999 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.417522+0000", 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5843000000000001e-05 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.418848+0000", 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013263459999999999 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(3.13 66/65 [PushOp(3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f, version: 44'54, data_included: [2317796~41500,2621440~131072,2818048~131072,3014656~65536,3211264~131072,3407872~65536,3735552~131072,3932160~131072,4128768~131072,4325376~89572], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:cf6f6a6c:::rbd_data.117c3ee69d5.0000000000000000:f@44'54, size: 7732453, copy_subset: [(0, 7732453)], clone_subset: {}, snapset: f=[]:{f: [f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4414948, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2317796, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.421474+0000", 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.33585129400000002, 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026851969999999998, 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.421474+0000", 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.421474+0000", 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.421476+0000", 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.824e-06 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.422422+0000", 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00094604699999999999 2024-09-22T01:01:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.422454+0000", 2024-09-22T01:01:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2228999999999999e-05 2024-09-22T01:01:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.422481+0000", 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7384000000000001e-05 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.422679+0000", 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000197758 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.422694+0000", 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5304000000000001e-05 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.424159+0000", 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001464651 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:53802 3.13 e66/65 3:cb3ff6d1:::rbd_data.117c93605fe7.00000000000000ed:head v 66'110, mlcod=0'0)", 2024-09-22T01:01:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.32006452899999999, 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003122391, 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437261+0000", 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4799999999999998e-07 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437266+0000", 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437276+0000", 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6420000000000002e-06 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437283+0000", 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9450000000000004e-06 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.438702+0000", 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014196409999999999 2024-09-22T01:01:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.438725+0000", 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2456e-05 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.440350+0000", 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001624955 2024-09-22T01:01:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.440383+0000", 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3204999999999998e-05 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(3.13 66/65 [PushOp(3:cf517187:::rbd_data.117c83c8bd76.0000000000000000:1a, version: 62'106, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:cf517187:::rbd_data.117c83c8bd76.0000000000000000:1a@62'106, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1a=[]:{1a: [1a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479016+0000", 2024-09-22T01:01:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.27830937300000003, 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027362430000000002, 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479016+0000", 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479016+0000", 2024-09-22T01:01:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479016+0000", 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.480181+0000", 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001164368 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.480210+0000", 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9465e-05 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.480232+0000", 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1801999999999999e-05 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.480372+0000", 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013947499999999999 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.480386+0000", 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4552999999999999e-05 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.481752+0000", 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00136578 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(3.13 66/65 [PushOp(3:cf517187:::rbd_data.117c83c8bd76.0000000000000000:1a, version: 62'106, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:cf517187:::rbd_data.117c83c8bd76.0000000000000000:1a@62'106, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1a=[]:{1a: [1a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-22T01:01:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.482895+0000", 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.27443041400000001, 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027076779999999998, 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482895+0000", 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482895+0000", 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482895+0000", 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3200000000000004e-07 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.483990+0000", 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010945589999999999 2024-09-22T01:01:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.484016+0000", 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5734000000000001e-05 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.484038+0000", 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2325999999999999e-05 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.484150+0000", 2024-09-22T01:01:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112333 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.484167+0000", 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6311e-05 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.485602+0000", 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001435583 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:41.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:41.853 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:41.853 INFO:teuthology.orchestra.run.smithi002.stdout:2566 write 0x5760130 thru 0x576031c (0x1ed bytes) 2024-09-22T01:01:41.854 INFO:teuthology.orchestra.run.smithi002.stdout:2567 punch from 0xbd3af9 to 0xbd5693, (0x1b9a bytes) 2024-09-22T01:01:41.854 INFO:teuthology.orchestra.run.smithi002.stdout:2568 write 0x7ba285c thru 0x7bad685 (0xae2a bytes) 2024-09-22T01:01:41.855 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:41.856 INFO:teuthology.orchestra.run.smithi002.stdout:2569 read 0xb4eec04 thru 0xb4f2242 (0x363f bytes) 2024-09-22T01:01:41.856 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:41.856 INFO:teuthology.orchestra.run.smithi002.stdout:2572 punch from 0x46f609 to 0x478e9e, (0x9895 bytes) 2024-09-22T01:01:41.856 INFO:teuthology.orchestra.run.smithi002.stdout:2573 write 0x65f07a thru 0x665225 (0x61ac bytes) 2024-09-22T01:01:41.856 INFO:teuthology.orchestra.run.smithi002.stdout:2574 read 0x72ccc00 thru 0x72d4f30 (0x8331 bytes) 2024-09-22T01:01:41.856 INFO:teuthology.orchestra.run.smithi002.stdout:2576 read 0x761b6bb thru 0x7620c2f (0x5575 bytes) 2024-09-22T01:01:41.856 INFO:teuthology.orchestra.run.smithi002.stdout:2578 writesame 0x877fe85 thru 0x878ec1e data_size 0x11(0xed9a bytes) 2024-09-22T01:01:41.857 INFO:teuthology.orchestra.run.smithi002.stdout:2579 write 0x676d857 thru 0x677c782 (0xef2c bytes) 2024-09-22T01:01:41.857 INFO:teuthology.orchestra.run.smithi002.stdout:2580 write 0xbac3c94 thru 0xbad2992 (0xecff bytes) 2024-09-22T01:01:41.857 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:41.851+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:41.866 INFO:teuthology.orchestra.run.smithi002.stdout:2582 trunc from 0xe32510f to 0x15af393 2024-09-22T01:01:41.942 INFO:teuthology.orchestra.run.smithi002.stdout:2583 write 0xd22a3f8 thru 0xd233ba1 (0x97aa bytes) 2024-09-22T01:01:41.955 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:41.952+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:41.960 INFO:teuthology.orchestra.run.smithi002.stdout:2584 read 0x7fd9998 thru 0x7fded39 (0x53a2 bytes) 2024-09-22T01:01:41.960 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:41.968 INFO:teuthology.orchestra.run.smithi002.stdout:2586 writesame 0xda6769f thru 0xda6ad67 data_size 0x37(0x36c9 bytes) 2024-09-22T01:01:41.979 INFO:teuthology.orchestra.run.smithi002.stdout:2587 trunc from 0xda6ad68 to 0x39fe23b 2024-09-22T01:01:41.995 INFO:teuthology.orchestra.run.smithi002.stdout:2588 read 0x101e5c4 thru 0x102768d (0x90ca bytes) 2024-09-22T01:01:41.997 INFO:teuthology.orchestra.run.smithi002.stdout:2589 read 0x3398fdd thru 0x339cf94 (0x3fb8 bytes) 2024-09-22T01:01:41.997 INFO:teuthology.orchestra.run.smithi002.stdout:2590 read 0x1f7f695 thru 0x1f86970 (0x72dc bytes) 2024-09-22T01:01:42.004 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:42.004 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:42.017 INFO:teuthology.orchestra.run.smithi002.stdout:2591 writesame 0xe1c3373 thru 0xe1cec36 data_size 0xac(0xb8c4 bytes) 2024-09-22T01:01:42.018 INFO:teuthology.orchestra.run.smithi002.stdout:2592 punch from 0x8d78ba8 to 0x8d825f5, (0x9a4d bytes) 2024-09-22T01:01:42.018 INFO:teuthology.orchestra.run.smithi002.stdout:2594 read 0x84542c1 thru 0x8463b10 (0xf850 bytes) 2024-09-22T01:01:42.018 INFO:teuthology.orchestra.run.smithi002.stdout:2595 punch from 0xd6632b to 0xd6f9e4, (0x96b9 bytes) 2024-09-22T01:01:42.018 INFO:teuthology.orchestra.run.smithi002.stdout:2597 read 0xd86a500 thru 0xd87223d (0x7d3e bytes) 2024-09-22T01:01:42.018 INFO:teuthology.orchestra.run.smithi002.stdout:2598 write 0x15fd72a thru 0x160707d (0x9954 bytes) 2024-09-22T01:01:42.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:42.035 INFO:teuthology.orchestra.run.smithi002.stdout:2599 read 0x4a8e1da thru 0x4a94ebf (0x6ce6 bytes) 2024-09-22T01:01:42.035 INFO:teuthology.orchestra.run.smithi002.stdout:2600 write 0xaaaf273 thru 0xaab11b0 (0x1f3e bytes) 2024-09-22T01:01:42.043 INFO:teuthology.orchestra.run.smithi002.stdout:2601 read 0x7336684 thru 0x7337065 (0x9e2 bytes) 2024-09-22T01:01:42.043 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:42.043 INFO:teuthology.orchestra.run.smithi002.stdout:2603 write 0xa878619 thru 0xa8870ab (0xea93 bytes) 2024-09-22T01:01:42.056 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:42.052+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:42.057 INFO:teuthology.orchestra.run.smithi002.stdout:2604 write 0xb70a81 thru 0xb7d534 (0xcab4 bytes) 2024-09-22T01:01:42.063 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:42.063 INFO:teuthology.orchestra.run.smithi002.stdout:2607 read 0xab81487 thru 0xab85040 (0x3bba bytes) 2024-09-22T01:01:42.072 INFO:teuthology.orchestra.run.smithi002.stdout:2608 trunc from 0xe1cec37 to 0x6ded805 2024-09-22T01:01:42.090 INFO:teuthology.orchestra.run.smithi002.stdout:2609 read 0x41e1e37 thru 0x41e6d1d (0x4ee7 bytes) 2024-09-22T01:01:42.092 INFO:teuthology.orchestra.run.smithi002.stdout:2611 trunc from 0x6ded805 to 0x4dcf4f5 2024-09-22T01:01:42.102 INFO:teuthology.orchestra.run.smithi002.stdout:2613 punch from 0x2484139 to 0x2487786, (0x364d bytes) 2024-09-22T01:01:42.102 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:42.102 INFO:teuthology.orchestra.run.smithi002.stdout:2615 read 0x35ca7f7 thru 0x35ce60a (0x3e14 bytes) 2024-09-22T01:01:42.110 INFO:teuthology.orchestra.run.smithi002.stdout:2617 trunc from 0x4dcf4f5 to 0xc6951d9 2024-09-22T01:01:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:42.156 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:42.151+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:42.165 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:42.188 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:42.257 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:42.256+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86072121499999998, 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:01:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86070038599999998, 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:01:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86067187099999998, 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86064922099999996, 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86062722599999997, 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:01:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86060216199999995, 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:01:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86057972900000002, 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:01:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:01:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86055825200000002, 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86052949400000001, 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:01:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85971650200000005, 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85969181699999997, 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:01:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:01:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85537028299999995, 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85534143200000001, 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:01:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:01:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85531970700000004, 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85529927699999997, 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:01:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:01:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85527696099999995, 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:01:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85520916599999997, 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:01:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.855176029, 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85515210699999999, 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:01:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.85512851000000001, 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:42.424 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:42.425 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:42.425 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:42.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:42.427 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:42.427 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:42.427 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:42.428 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:42.428 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:42.428 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:42.428 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:42.428 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:42.429 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:42.423+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:42.431 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:42.431 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:42.450 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:42.450 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:42.467 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:42.467 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:42.526 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:42.524+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:42.559 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:42.559 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:42.573 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:42.660 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:42.660+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:42.728 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:42.728+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:42.740 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:42.740 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:42.754 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:42.840 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:42.840+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:42.915 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:42.915 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:42.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:42.930 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:42.928+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:43.030 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:43.028+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:43.097 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:43.097 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:43.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:43.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:01:43.144 INFO:teuthology.orchestra.run.smithi002.stdout:2618 write 0x6af60e3 thru 0x6afabdf (0x4afd bytes) 2024-09-22T01:01:43.156 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:43.155+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:43.170 INFO:teuthology.orchestra.run.smithi002.stdout:2619 punch from 0x54ac6e8 to 0x54b8111, (0xba29 bytes) 2024-09-22T01:01:43.170 INFO:teuthology.orchestra.run.smithi002.stdout:2620 read 0x739a616 thru 0x73a2424 (0x7e0f bytes) 2024-09-22T01:01:43.170 INFO:teuthology.orchestra.run.smithi002.stdout:2621 read 0x54f9c0b thru 0x55054ed (0xb8e3 bytes) 2024-09-22T01:01:43.170 INFO:teuthology.orchestra.run.smithi002.stdout:2622 writesame 0x68af81c thru 0x68bde2f data_size 0x7c(0xe614 bytes) 2024-09-22T01:01:43.196 INFO:teuthology.orchestra.run.smithi002.stdout:2624 write 0x20b0345 thru 0x20b316a (0x2e26 bytes) 2024-09-22T01:01:43.214 INFO:teuthology.orchestra.run.smithi002.stdout:2625 trunc from 0xc6951d9 to 0xe24054c 2024-09-22T01:01:43.231 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:43.227+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:43.236 INFO:teuthology.orchestra.run.smithi002.stdout:2626 trunc from 0xe24054c to 0x53f5c24 2024-09-22T01:01:43.257 INFO:teuthology.orchestra.run.smithi002.stdout:2630 write 0x5360147 thru 0x536c741 (0xc5fb bytes) 2024-09-22T01:01:43.281 INFO:teuthology.orchestra.run.smithi002.stdout:2631 read 0x2aa04c6 thru 0x2aa533f (0x4e7a bytes) 2024-09-22T01:01:43.290 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:43.290 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:2633 write 0xbf9cd4f thru 0xbfa5564 (0x8816 bytes) 2024-09-22T01:01:43.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:43.318 INFO:teuthology.orchestra.run.smithi002.stdout:2634 read 0x2b24ffb thru 0x2b3212b (0xd131 bytes) 2024-09-22T01:01:43.318 INFO:teuthology.orchestra.run.smithi002.stdout:2635 read 0x2da6de8 thru 0x2daad08 (0x3f21 bytes) 2024-09-22T01:01:43.318 INFO:teuthology.orchestra.run.smithi002.stdout:2637 read 0x35f00f3 thru 0x35ffe3f (0xfd4d bytes) 2024-09-22T01:01:43.318 INFO:teuthology.orchestra.run.smithi002.stdout:2638 writesame 0x8c1c1ec thru 0x8c2078d data_size 0x3(0x45a2 bytes) 2024-09-22T01:01:43.332 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:43.327+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:43.343 INFO:teuthology.orchestra.run.smithi002.stdout:2640 writesame 0xd6ae12b thru 0xd6b5a97 data_size 0x1(0x796d bytes) 2024-09-22T01:01:43.343 INFO:teuthology.orchestra.run.smithi002.stdout:2641 read 0xc3e2190 thru 0xc3ecfd6 (0xae47 bytes) 2024-09-22T01:01:43.343 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.343 INFO:teuthology.orchestra.run.smithi002.stdout:2644 write 0x5113aaf thru 0x5116fc6 (0x3518 bytes) 2024-09-22T01:01:43.346 INFO:teuthology.orchestra.run.smithi002.stdout:2645 writesame 0x9a87146 thru 0x9a93788 data_size 0x1(0xc643 bytes) 2024-09-22T01:01:43.368 INFO:teuthology.orchestra.run.smithi002.stdout:2646 write 0x3b31ef0 thru 0x3b3ebbe (0xcccf bytes) 2024-09-22T01:01:43.386 INFO:teuthology.orchestra.run.smithi002.stdout:2647 flatten 2024-09-22T01:01:43.402 INFO:teuthology.orchestra.run.smithi002.stdout:2650 trunc from 0xd6b5a98 to 0x865d44e 2024-09-22T01:01:43.428 INFO:teuthology.orchestra.run.smithi002.stdout:2651 read 0x2f7b041 thru 0x2f7bdc5 (0xd85 bytes) 2024-09-22T01:01:43.428 INFO:teuthology.orchestra.run.smithi002.stdout:2652 read 0x59c4ec9 thru 0x59cadf6 (0x5f2e bytes) 2024-09-22T01:01:43.428 INFO:teuthology.orchestra.run.smithi002.stdout:2653 punch from 0x6183e17 to 0x618e6b8, (0xa8a1 bytes) 2024-09-22T01:01:43.428 INFO:teuthology.orchestra.run.smithi002.stdout:2656 read 0x74f1694 thru 0x74f2b35 (0x14a2 bytes) 2024-09-22T01:01:43.429 INFO:teuthology.orchestra.run.smithi002.stdout:2657 punch from 0x5292f55 to 0x529b388, (0x8433 bytes) 2024-09-22T01:01:43.429 INFO:teuthology.orchestra.run.smithi002.stdout:2658 writesame 0x66ba532 thru 0x66c316e data_size 0x3(0x8c3d bytes) 2024-09-22T01:01:43.433 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:43.431+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:43.448 INFO:teuthology.orchestra.run.smithi002.stdout:2659 writesame 0x7617b77 thru 0x761b09e data_size 0x54(0x3528 bytes) 2024-09-22T01:01:43.468 INFO:teuthology.orchestra.run.smithi002.stdout:2661 write 0x90fa563 thru 0x9107d2e (0xd7cc bytes) 2024-09-22T01:01:43.486 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:43.487 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:43.487 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.487 INFO:teuthology.orchestra.run.smithi002.stdout:2664 punch from 0x71496f6 to 0x714ec1e, (0x5528 bytes) 2024-09-22T01:01:43.487 INFO:teuthology.orchestra.run.smithi002.stdout:2665 writesame 0x3818cb6 thru 0x3820c55 data_size 0x10(0x7fa0 bytes) 2024-09-22T01:01:43.492 INFO:teuthology.orchestra.run.smithi002.stdout:2667 read 0x25c4d69 thru 0x25cb9cb (0x6c63 bytes) 2024-09-22T01:01:43.492 INFO:teuthology.orchestra.run.smithi002.stdout:2668 read 0x3ac335a thru 0x3ad1348 (0xdfef bytes) 2024-09-22T01:01:43.494 INFO:teuthology.orchestra.run.smithi002.stdout:2669 punch from 0x8432f8f to 0x8441938, (0xe9a9 bytes) 2024-09-22T01:01:43.494 INFO:teuthology.orchestra.run.smithi002.stdout:2670 write 0x8db6684 thru 0x8db7ad1 (0x144e bytes) 2024-09-22T01:01:43.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:43.507 INFO:teuthology.orchestra.run.smithi002.stdout:2671 writesame 0x92e72aa thru 0x92ec16f data_size 0x3(0x4ec6 bytes) 2024-09-22T01:01:43.518 INFO:teuthology.orchestra.run.smithi002.stdout:2672 write 0xa267e77 thru 0xa2728ae (0xaa38 bytes) 2024-09-22T01:01:43.536 INFO:teuthology.orchestra.run.smithi002.stdout:2673 write 0x17a9452 thru 0x17b1a97 (0x8646 bytes) 2024-09-22T01:01:43.541 INFO:teuthology.orchestra.run.smithi002.stdout:2675 punch from 0x7022e59 to 0x7027090, (0x4237 bytes) 2024-09-22T01:01:43.541 INFO:teuthology.orchestra.run.smithi002.stdout:2676 write 0x35a7189 thru 0x35b6ffd (0xfe75 bytes) 2024-09-22T01:01:43.548 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:43.547+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:43.561 INFO:teuthology.orchestra.run.smithi002.stdout:2677 trunc from 0xa2728af to 0x93fbb58 2024-09-22T01:01:43.575 INFO:teuthology.orchestra.run.smithi002.stdout:2678 read 0x6e0055b thru 0x6e04840 (0x42e6 bytes) 2024-09-22T01:01:43.575 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.575 INFO:teuthology.orchestra.run.smithi002.stdout:2682 write 0x681c592 thru 0x682bbe5 (0xf654 bytes) 2024-09-22T01:01:43.588 INFO:teuthology.orchestra.run.smithi002.stdout:2683 writesame 0xa44f2df thru 0xa459bc2 data_size 0x12(0xa8e4 bytes) 2024-09-22T01:01:43.601 INFO:teuthology.orchestra.run.smithi002.stdout:2684 writesame 0xe61bf5a thru 0xe629a9e data_size 0x51(0xdb45 bytes) 2024-09-22T01:01:43.602 INFO:teuthology.orchestra.run.smithi002.stdout:2685 write 0x31a60e9 thru 0x31b1f1f (0xbe37 bytes) 2024-09-22T01:01:43.635 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:43.631+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:43.636 INFO:teuthology.orchestra.run.smithi002.stdout:2686 trunc from 0xe629a9f to 0x2e014dc 2024-09-22T01:01:43.688 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:43.688 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:43.688 INFO:teuthology.orchestra.run.smithi002.stdout:2688 write 0x42da7c9 thru 0x42e39d0 (0x9208 bytes) 2024-09-22T01:01:43.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:43.707 INFO:teuthology.orchestra.run.smithi002.stdout:2689 punch from 0x3d90e90 to 0x3d9e284, (0xd3f4 bytes) 2024-09-22T01:01:43.707 INFO:teuthology.orchestra.run.smithi002.stdout:2690 punch from 0x211eb07 to 0x212b903, (0xcdfc bytes) 2024-09-22T01:01:43.718 INFO:teuthology.orchestra.run.smithi002.stdout:2691 write 0x67e1f01 thru 0x67ed497 (0xb597 bytes) 2024-09-22T01:01:43.735 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:43.731+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:43.739 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.741 INFO:teuthology.orchestra.run.smithi002.stdout:2694 trunc from 0x67ed498 to 0x5beef56 2024-09-22T01:01:43.755 INFO:teuthology.orchestra.run.smithi002.stdout:2696 punch from 0x1c672ac to 0x1c743fd, (0xd151 bytes) 2024-09-22T01:01:43.756 INFO:teuthology.orchestra.run.smithi002.stdout:2697 read 0x49188ec thru 0x4918e9a (0x5af bytes) 2024-09-22T01:01:43.756 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.756 INFO:teuthology.orchestra.run.smithi002.stdout:2699 read 0x4bf7af3 thru 0x4bffe86 (0x8394 bytes) 2024-09-22T01:01:43.756 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.756 INFO:teuthology.orchestra.run.smithi002.stdout:2701 punch from 0x42dc4c6 to 0x42ea5f7, (0xe131 bytes) 2024-09-22T01:01:43.767 INFO:teuthology.orchestra.run.smithi002.stdout:2702 trunc from 0x5beef56 to 0xcef3fce 2024-09-22T01:01:43.774 INFO:teuthology.orchestra.run.smithi002.stdout:2703 write 0x31984fc thru 0x3199b6a (0x166f bytes) 2024-09-22T01:01:43.780 INFO:teuthology.orchestra.run.smithi002.stdout:2704 write 0xab0f922 thru 0xab139ed (0x40cc bytes) 2024-09-22T01:01:43.794 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.811 INFO:teuthology.orchestra.run.smithi002.stdout:2711 trunc from 0xcef3fce to 0x1fe199f 2024-09-22T01:01:43.837 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:43.831+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:43.839 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.840 INFO:teuthology.orchestra.run.smithi002.stdout:2714 read 0x4d6a7d thru 0x4e40f6 (0xd67a bytes) 2024-09-22T01:01:43.843 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:43.843 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:43.857 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:43.872 INFO:teuthology.orchestra.run.smithi002.stdout:2715 write 0xe270e80 thru 0xe27693e (0x5abf bytes) 2024-09-22T01:01:43.896 INFO:teuthology.orchestra.run.smithi002.stdout:2718 punch from 0x1a70c1c to 0x1a75892, (0x4c76 bytes) 2024-09-22T01:01:43.896 INFO:teuthology.orchestra.run.smithi002.stdout:2719 write 0x19dd2b6 thru 0x19dec06 (0x1951 bytes) 2024-09-22T01:01:43.905 INFO:teuthology.orchestra.run.smithi002.stdout:2720 write 0xd3cc9b5 thru 0xd3cedcb (0x2417 bytes) 2024-09-22T01:01:43.915 INFO:teuthology.orchestra.run.smithi002.stdout:2721 writesame 0x8b1ccaf thru 0x8b1d392 data_size 0x15(0x6e4 bytes) 2024-09-22T01:01:43.920 INFO:teuthology.orchestra.run.smithi002.stdout:2724 read 0xd90d629 thru 0xd912331 (0x4d09 bytes) 2024-09-22T01:01:43.920 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.921 INFO:teuthology.orchestra.run.smithi002.stdout:2727 trunc from 0xe27693f to 0xdb6e102 2024-09-22T01:01:43.938 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:43.935+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:43.939 INFO:teuthology.orchestra.run.smithi002.stdout:2728 read 0x724db55 thru 0x7250cd1 (0x317d bytes) 2024-09-22T01:01:43.939 INFO:teuthology.orchestra.run.smithi002.stdout:2729 read 0x9b4127c thru 0x9b4e813 (0xd598 bytes) 2024-09-22T01:01:43.939 INFO:teuthology.orchestra.run.smithi002.stdout:2730 write 0x2e9970f thru 0x2e9c969 (0x325b bytes) 2024-09-22T01:01:43.953 INFO:teuthology.orchestra.run.smithi002.stdout:2731 read 0x1fe8269 thru 0x1fefabe (0x7856 bytes) 2024-09-22T01:01:43.954 INFO:teuthology.orchestra.run.smithi002.stdout:2732 write 0x53d675c thru 0x53d7fd0 (0x1875 bytes) 2024-09-22T01:01:43.962 INFO:teuthology.orchestra.run.smithi002.stdout:2733 writesame 0x6c0a316 thru 0x6c185db data_size 0x1(0xe2c6 bytes) 2024-09-22T01:01:43.992 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:43.992 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:2736 write 0x12c10d5 thru 0x12c3bdc (0x2b08 bytes) 2024-09-22T01:01:44.004 INFO:teuthology.orchestra.run.smithi002.stdout:2737 punch from 0xd185b39 to 0xd1921a0, (0xc667 bytes) 2024-09-22T01:01:44.004 INFO:teuthology.orchestra.run.smithi002.stdout:2738 writesame 0xbff038c thru 0xbff692c data_size 0x1(0x65a1 bytes) 2024-09-22T01:01:44.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:44.027 INFO:teuthology.orchestra.run.smithi002.stdout:2740 punch from 0x44a7ddf to 0x44af90a, (0x7b2b bytes) 2024-09-22T01:01:44.027 INFO:teuthology.orchestra.run.smithi002.stdout:2741 punch from 0x2f98945 to 0x2fa82f6, (0xf9b1 bytes) 2024-09-22T01:01:44.027 INFO:teuthology.orchestra.run.smithi002.stdout:2742 read 0x1dd424 thru 0x1e1957 (0x4534 bytes) 2024-09-22T01:01:44.027 INFO:teuthology.orchestra.run.smithi002.stdout:2744 read 0xd0846a8 thru 0xd084b3e (0x497 bytes) 2024-09-22T01:01:44.039 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:44.035+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:44.041 INFO:teuthology.orchestra.run.smithi002.stdout:2746 trunc from 0xdb6e102 to 0x4fec984 2024-09-22T01:01:44.073 INFO:teuthology.orchestra.run.smithi002.stdout:2747 trunc from 0x4fec984 to 0xcb0cf70 2024-09-22T01:01:44.083 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.083 INFO:teuthology.orchestra.run.smithi002.stdout:2752 write 0xa1e8ea thru 0xa28aba (0xa1d1 bytes) 2024-09-22T01:01:44.104 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.104 INFO:teuthology.orchestra.run.smithi002.stdout:2754 punch from 0x2d48c96 to 0x2d55f1a, (0xd284 bytes) 2024-09-22T01:01:44.104 INFO:teuthology.orchestra.run.smithi002.stdout:2755 read 0x1f4f7f4 thru 0x1f504e6 (0xcf3 bytes) 2024-09-22T01:01:44.104 INFO:teuthology.orchestra.run.smithi002.stdout:2756 writesame 0x33b1bdd thru 0x33b8342 data_size 0x5(0x6766 bytes) 2024-09-22T01:01:44.107 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:01:44.108 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:01:44.108 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:01:44.123 INFO:teuthology.orchestra.run.smithi002.stdout:2757 read 0x44889e5 thru 0x4495793 (0xcdaf bytes) 2024-09-22T01:01:44.123 INFO:teuthology.orchestra.run.smithi002.stdout:2759 writesame 0xaecaeea thru 0xaed6596 data_size 0x2f(0xb6ad bytes) 2024-09-22T01:01:44.133 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:01:44.134 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:44.134 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:44.140 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:44.135+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:44.148 INFO:teuthology.orchestra.run.smithi002.stdout:2760 writesame 0x5660d02 thru 0x56640df data_size 0x3(0x33de bytes) 2024-09-22T01:01:44.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:01:44.162 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.162 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.162 INFO:teuthology.orchestra.run.smithi002.stdout:2763 trunc from 0xcb0cf70 to 0xcbef912 2024-09-22T01:01:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:2764 write 0xda93a03 thru 0xda98945 (0x4f43 bytes) 2024-09-22T01:01:44.192 INFO:teuthology.orchestra.run.smithi002.stdout:2765 punch from 0x1927d0d to 0x19325e0, (0xa8d3 bytes) 2024-09-22T01:01:44.200 INFO:teuthology.orchestra.run.smithi002.stdout:2766 trunc from 0xda98946 to 0x79ff097 2024-09-22T01:01:44.237 INFO:teuthology.orchestra.run.smithi002.stdout:2767 writesame 0xe15a96b thru 0xe167f37 data_size 0x7(0xd5cd bytes) 2024-09-22T01:01:44.237 INFO:teuthology.orchestra.run.smithi002.stdout:2769 read 0xcab6e09 thru 0xcabf2be (0x84b6 bytes) 2024-09-22T01:01:44.237 INFO:teuthology.orchestra.run.smithi002.stdout:2770 write 0x847b695 thru 0x84830f2 (0x7a5e bytes) 2024-09-22T01:01:44.240 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:44.235+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:44.256 INFO:teuthology.orchestra.run.smithi002.stdout:2772 write 0xc558c90 thru 0xc562476 (0x97e7 bytes) 2024-09-22T01:01:44.274 INFO:teuthology.orchestra.run.smithi002.stdout:2773 read 0x86ff2f9 thru 0x870b218 (0xbf20 bytes) 2024-09-22T01:01:44.274 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.275 INFO:teuthology.orchestra.run.smithi002.stdout:2775 trunc from 0xe167f38 to 0xd8a6abe 2024-09-22T01:01:44.284 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.285 INFO:teuthology.orchestra.run.smithi002.stdout:2779 write 0x6ee58f0 thru 0x6ef4ab3 (0xf1c4 bytes) 2024-09-22T01:01:44.285 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:44.285 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:44.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:01:44.306 INFO:teuthology.orchestra.run.smithi002.stdout:2780 read 0xacf8726 thru 0xacfa8c4 (0x219f bytes) 2024-09-22T01:01:44.306 INFO:teuthology.orchestra.run.smithi002.stdout:2781 write 0x9fdd0c9 thru 0x9fe6a06 (0x993e bytes) 2024-09-22T01:01:44.327 INFO:teuthology.orchestra.run.smithi002.stdout:2783 writesame 0x6c0a05c thru 0x6c11ae0 data_size 0x55(0x7a85 bytes) 2024-09-22T01:01:44.342 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:44.335+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:44.350 INFO:teuthology.orchestra.run.smithi002.stdout:2785 trunc from 0xd8a6abe to 0xb2bdd36 2024-09-22T01:01:44.365 INFO:teuthology.orchestra.run.smithi002.stdout:2787 read 0x243e8f0 thru 0x244ba06 (0xd117 bytes) 2024-09-22T01:01:44.371 INFO:teuthology.orchestra.run.smithi002.stdout:2788 trunc from 0xb2bdd36 to 0x72b0090 2024-09-22T01:01:44.408 INFO:teuthology.orchestra.run.smithi002.stdout:2789 write 0xce70455 thru 0xce70d38 (0x8e4 bytes) 2024-09-22T01:01:44.412 INFO:teuthology.orchestra.run.smithi002.stdout:2790 read 0x2a16e2d thru 0x2a1c9a5 (0x5b79 bytes) 2024-09-22T01:01:44.412 INFO:teuthology.orchestra.run.smithi002.stdout:2791 write 0x11f0e0e thru 0x11f3534 (0x2727 bytes) 2024-09-22T01:01:44.418 INFO:teuthology.orchestra.run.smithi002.stdout:2792 write 0x2baf7e0 thru 0x2bb21db (0x29fc bytes) 2024-09-22T01:01:44.430 INFO:teuthology.orchestra.run.smithi002.stdout:2793 write 0x7078111 thru 0x7082f6d (0xae5d bytes) 2024-09-22T01:01:44.434 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:44.434 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:44.442 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:44.439+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:44.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:44.451 INFO:teuthology.orchestra.run.smithi002.stdout:2795 punch from 0xcb89660 to 0xcb8e923, (0x52c3 bytes) 2024-09-22T01:01:44.451 INFO:teuthology.orchestra.run.smithi002.stdout:2796 punch from 0xa44e151 to 0xa45179d, (0x364c bytes) 2024-09-22T01:01:44.456 INFO:teuthology.orchestra.run.smithi002.stdout:2798 trunc from 0xce70d39 to 0x9d82f1a 2024-09-22T01:01:44.469 INFO:teuthology.orchestra.run.smithi002.stdout:2799 read 0x7a3f765 thru 0x7a439b9 (0x4255 bytes) 2024-09-22T01:01:44.469 INFO:teuthology.orchestra.run.smithi002.stdout:2800 read 0x6380969 thru 0x638217c (0x1814 bytes) 2024-09-22T01:01:44.469 INFO:teuthology.orchestra.run.smithi002.stdout:2802 read 0x5f70171 thru 0x5f7b1ea (0xb07a bytes) 2024-09-22T01:01:44.472 INFO:teuthology.orchestra.run.smithi002.stdout:2803 trunc from 0x9d82f1a to 0x8125655 2024-09-22T01:01:44.479 INFO:teuthology.orchestra.run.smithi002.stdout:2805 read 0x5e69c27 thru 0x5e6a61b (0x9f5 bytes) 2024-09-22T01:01:44.479 INFO:teuthology.orchestra.run.smithi002.stdout:2807 writesame 0x1f2f121 thru 0x1f3be28 data_size 0x36(0xcd08 bytes) 2024-09-22T01:01:44.484 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:44.503 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.503 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.503 INFO:teuthology.orchestra.run.smithi002.stdout:2811 read 0x1d64448 thru 0x1d68a5a (0x4613 bytes) 2024-09-22T01:01:44.504 INFO:teuthology.orchestra.run.smithi002.stdout:2812 read 0x7bbac2e thru 0x7bbedfa (0x41cd bytes) 2024-09-22T01:01:44.504 INFO:teuthology.orchestra.run.smithi002.stdout:2813 punch from 0x7073199 to 0x70779ae, (0x4815 bytes) 2024-09-22T01:01:44.504 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.504 INFO:teuthology.orchestra.run.smithi002.stdout:2815 read 0x666779c thru 0x6675e4d (0xe6b2 bytes) 2024-09-22T01:01:44.504 INFO:teuthology.orchestra.run.smithi002.stdout:2817 write 0x7144188 thru 0x7145750 (0x15c9 bytes) 2024-09-22T01:01:44.506 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.506 INFO:teuthology.orchestra.run.smithi002.stdout:2822 read 0x378cab3 thru 0x379420e (0x775c bytes) 2024-09-22T01:01:44.506 INFO:teuthology.orchestra.run.smithi002.stdout:2824 writesame 0x38ace91 thru 0x38b6d7b data_size 0x47(0x9eeb bytes) 2024-09-22T01:01:44.520 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:01:44.521 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:01:44.532 INFO:teuthology.orchestra.run.smithi002.stdout:2825 read 0x2f1f920 thru 0x2f285d6 (0x8cb7 bytes) 2024-09-22T01:01:44.533 INFO:teuthology.orchestra.run.smithi002.stdout:2826 write 0x4f7fe69 thru 0x4f88d47 (0x8edf bytes) 2024-09-22T01:01:44.543 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:44.539+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:44.553 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-22T01:01:44.131954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":608276052,"num_objects":949,"num_object_clones":261,"num_object_copies":1898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":949,"num_whiteouts":394,"num_read":73089,"num_read_kb":274871,"num_write":7648,"num_write_kb":174320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":28373038,"num_keys_recovered":146,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6337,"ondisk_log_size":6337,"up":82,"acting":69,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":358096,"kb_used_data":122028,"kb_used_omap":13,"kb_used_meta":235890,"kb_avail":660244784,"statfs":{"total":676457349120,"available":676090658816,"internally_reserved":0,"allocated":124956672,"data_stored":144998555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":241551604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":63701123,"num_objects":100,"num_object_clones":27,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":14,"num_read":10255,"num_read_kb":9604,"num_write":737,"num_write_kb":5601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":167229,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.055509"},"pg_stats":[{"pgid":"3.1f","version":"63'188","reported_seq":919,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.078896+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:01:40.078896+0000","last_peered":"2024-09-22T01:01:40.078896+0000","last_clean":"2024-09-22T01:01:40.078896+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:01:40.078896+0000","last_undegraded":"2024-09-22T01:01:40.078896+0000","last_fullsized":"2024-09-22T01:01:40.078896+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:14.726578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":26189761,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":834,"num_read_kb":9165,"num_write":226,"num_write_kb":6847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'318","reported_seq":1942,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.081919+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:01:40.081919+0000","last_peered":"2024-09-22T01:01:40.081919+0000","last_clean":"2024-09-22T01:01:40.081919+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:01:40.081919+0000","last_undegraded":"2024-09-22T01:01:40.081919+0000","last_fullsized":"2024-09-22T01:01:40.081919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:33:47.599150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":21981324,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":3661,"num_read_kb":8800,"num_write":421,"num_write_kb":5469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"63'181","reported_seq":2275,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:39.088332+0000","last_change":"2024-09-22T01:01:40.096952+0000","last_active":"2024-09-22T01:01:39.088332+0000","last_peered":"2024-09-22T01:01:39.075114+0000","last_clean":"2024-09-22T01:01:39.075114+0000","last_became_active":"2024-09-22T01:01:31.018372+0000","last_became_peered":"2024-09-22T01:01:31.018372+0000","last_unstale":"2024-09-22T01:01:39.088332+0000","last_undegraded":"2024-09-22T01:01:39.088332+0000","last_fullsized":"2024-09-22T01:01:39.088332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22219735,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":2948,"num_read_kb":13411,"num_write":184,"num_write_kb":10144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":3845944,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'443","reported_seq":2813,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:39.087263+0000","last_change":"2024-09-22T01:01:40.091319+0000","last_active":"2024-09-22T01:01:39.087263+0000","last_peered":"2024-09-22T01:01:39.047585+0000","last_clean":"2024-09-22T01:01:39.047585+0000","last_became_active":"2024-09-22T01:01:31.019916+0000","last_became_peered":"2024-09-22T01:01:31.019916+0000","last_unstale":"2024-09-22T01:01:39.087263+0000","last_undegraded":"2024-09-22T01:01:39.087263+0000","last_fullsized":"2024-09-22T01:01:39.087263+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43799612,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":3962,"num_read_kb":28377,"num_write":716,"num_write_kb":9890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4532656,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"63'109","reported_seq":329,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:39.076183+0000","last_change":"2024-09-22T01:01:40.079990+0000","last_active":"2024-09-22T01:01:39.076183+0000","last_peered":"2024-09-22T01:01:38.975756+0000","last_clean":"2024-09-22T01:01:38.975756+0000","last_became_active":"2024-09-22T01:00:47.730798+0000","last_became_peered":"2024-09-22T01:00:47.730798+0000","last_unstale":"2024-09-22T01:01:39.076183+0000","last_undegraded":"2024-09-22T01:01:39.076183+0000","last_fullsized":"2024-09-22T01:01:39.076183+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":16096846,"num_objects":34,"num_object_clones":13,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":232,"num_read_kb":523,"num_write":103,"num_write_kb":517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"63'107","reported_seq":1549,"reported_epoch":65,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:40.077933+0000","last_change":"2024-09-22T01:01:40.077933+0000","last_active":"2024-09-22T01:01:39.076485+0000","last_peered":"2024-09-22T01:01:38.980769+0000","last_clean":"2024-09-22T01:01:38.980769+0000","last_became_active":"2024-09-22T01:00:47.730348+0000","last_became_peered":"2024-09-22T01:00:47.730348+0000","last_unstale":"2024-09-22T01:01:40.077933+0000","last_undegraded":"2024-09-22T01:01:40.077933+0000","last_fullsized":"2024-09-22T01:01:40.077933+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":20947010,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":1374,"num_read_kb":2196,"num_write":135,"num_write_kb":4587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"63'104","reported_seq":2941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.077274+0000","last_change":"2024-09-22T01:00:49.708786+0000","last_active":"2024-09-22T01:01:40.077274+0000","last_peered":"2024-09-22T01:01:40.077274+0000","last_clean":"2024-09-22T01:01:40.077274+0000","last_became_active":"2024-09-22T01:00:47.723339+0000","last_became_peered":"2024-09-22T01:00:47.723339+0000","last_unstale":"2024-09-22T01:01:40.077274+0000","last_undegraded":"2024-09-22T01:01:40.077274+0000","last_fullsized":"2024-09-22T01:01:40.077274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:26:20.566339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":23116851,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":2811,"num_read_kb":11812,"num_write":136,"num_write_kb":4655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"63'310","reported_seq":3746,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.081574+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:01:40.081574+0000","last_peered":"2024-09-22T01:01:40.081574+0000","last_clean":"2024-09-22T01:01:40.081574+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:01:40.081574+0000","last_undegraded":"2024-09-22T01:01:40.081574+0000","last_fullsized":"2024-09-22T01:01:40.081574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:52.534839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":38961485,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":4688,"num_read_kb":22220,"num_write":432,"num_write_kb":5996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"63'262","reported_seq":2143,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.079044+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:01:40.079044+0000","last_peered":"2024-09-22T01:01:40.079044+0000","last_clean":"2024-09-22T01:01:40.079044+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:01:40.079044+0000","last_undegraded":"2024-09-22T01:01:40.079044+0000","last_fullsized":"2024-09-22T01:01:40.079044+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:01.173264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":9905991,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":2880,"num_read_kb":5013,"num_write":355,"num_write_kb":1995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"63'107","reported_seq":175,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.091372+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:01:40.091372+0000","last_peered":"2024-09-22T01:01:40.091372+0000","last_clean":"2024-09-22T01:01:40.091372+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:01:40.091372+0000","last_undegraded":"2024-09-22T01:01:40.091372+0000","last_fullsized":"2024-09-22T01:01:40.091372+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:34.910189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12847320,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":1197,"num_read_kb":7277,"num_write":101,"num_write_kb":11373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"64'289","reported_seq":4539,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.086375+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:01:40.086375+0000","last_peered":"2024-09-22T01:01:40.086375+0000","last_clean":"2024-09-22T01:01:40.086375+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:01:40.086375+0000","last_undegraded":"2024-09-22T01:01:40.086375+0000","last_fullsized":"2024-09-22T01:01:40.086375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:27.759058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":19870675,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":5470,"num_read_kb":17800,"num_write":335,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"63'211","reported_seq":3069,"reported_epoch":65,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:40.087505+0000","last_change":"2024-09-22T01:01:40.087505+0000","last_active":"2024-09-22T01:01:39.081298+0000","last_peered":"2024-09-22T01:01:38.912095+0000","last_clean":"2024-09-22T01:01:38.912095+0000","last_became_active":"2024-09-22T01:00:47.735932+0000","last_became_peered":"2024-09-22T01:00:47.735932+0000","last_unstale":"2024-09-22T01:01:40.087505+0000","last_undegraded":"2024-09-22T01:01:40.087505+0000","last_fullsized":"2024-09-22T01:01:40.087505+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":23701546,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":3702,"num_read_kb":9198,"num_write":253,"num_write_kb":2696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":109,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-22T01:01:41.087600+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:01:41.087600+0000","last_peered":"2024-09-22T01:01:41.087600+0000","last_clean":"2024-09-22T01:01:41.087600+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:01:41.087600+0000","last_undegraded":"2024-09-22T01:01:41.087600+0000","last_fullsized":"2024-09-22T01:01:41.087600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"63'174","reported_seq":1437,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.078346+0000","last_change":"2024-09-22T01:00:49.729709+0000","last_active":"2024-09-22T01:01:40.078346+0000","last_peered":"2024-09-22T01:01:40.078346+0000","last_clean":"2024-09-22T01:01:40.078346+0000","last_became_active":"2024-09-22T01:00:47.718964+0000","last_became_peered":"2024-09-22T01:00:47.718964+0000","last_unstale":"2024-09-22T01:01:40.078346+0000","last_undegraded":"2024-09-22T01:01:40.078346+0000","last_fullsized":"2024-09-22T01:01:40.078346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:26:47.676106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":15804463,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":2833,"num_read_kb":4189,"num_write":160,"num_write_kb":1105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":107,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.078726+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:01:40.078726+0000","last_peered":"2024-09-22T01:01:40.078726+0000","last_clean":"2024-09-22T01:01:40.078726+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:01:40.078726+0000","last_undegraded":"2024-09-22T01:01:40.078726+0000","last_fullsized":"2024-09-22T01:01:40.078726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"63'311","reported_seq":1254,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.082024+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:01:40.082024+0000","last_peered":"2024-09-22T01:01:40.082024+0000","last_clean":"2024-09-22T01:01:40.082024+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:01:40.082024+0000","last_undegraded":"2024-09-22T01:01:40.082024+0000","last_fullsized":"2024-09-22T01:01:40.082024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:37.439769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":13606832,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":2002,"num_read_kb":6265,"num_write":469,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.086960+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:01:40.086960+0000","last_peered":"2024-09-22T01:01:40.086960+0000","last_clean":"2024-09-22T01:01:40.086960+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:01:40.086960+0000","last_undegraded":"2024-09-22T01:01:40.086960+0000","last_fullsized":"2024-09-22T01:01:40.086960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"63'145","reported_seq":2401,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.078366+0000","last_change":"2024-09-22T01:00:49.709328+0000","last_active":"2024-09-22T01:01:40.078366+0000","last_peered":"2024-09-22T01:01:40.078366+0000","last_clean":"2024-09-22T01:01:40.078366+0000","last_became_active":"2024-09-22T01:00:47.714621+0000","last_became_peered":"2024-09-22T01:00:47.714621+0000","last_unstale":"2024-09-22T01:01:40.078366+0000","last_undegraded":"2024-09-22T01:01:40.078366+0000","last_fullsized":"2024-09-22T01:01:40.078366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:25:53.877973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":9328669,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":2752,"num_read_kb":10991,"num_write":123,"num_write_kb":5549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":107,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.078496+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:01:40.078496+0000","last_peered":"2024-09-22T01:01:40.078496+0000","last_clean":"2024-09-22T01:01:40.078496+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:01:40.078496+0000","last_undegraded":"2024-09-22T01:01:40.078496+0000","last_fullsized":"2024-09-22T01:01:40.078496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"63'244","reported_seq":2258,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.140611+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:01:40.140611+0000","last_peered":"2024-09-22T01:01:40.140611+0000","last_clean":"2024-09-22T01:01:40.140611+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:01:40.140611+0000","last_undegraded":"2024-09-22T01:01:40.140611+0000","last_fullsized":"2024-09-22T01:01:40.140611+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:13:35.108288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12651832,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":4468,"num_read_kb":4293,"num_write":225,"num_write_kb":583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"63'178","reported_seq":653,"reported_epoch":65,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:40.079206+0000","last_change":"2024-09-22T01:01:40.079206+0000","last_active":"2024-09-22T01:01:39.075168+0000","last_peered":"2024-09-22T01:01:38.709960+0000","last_clean":"2024-09-22T01:01:38.709960+0000","last_became_active":"2024-09-22T01:00:47.721552+0000","last_became_peered":"2024-09-22T01:00:47.721552+0000","last_unstale":"2024-09-22T01:01:40.079206+0000","last_undegraded":"2024-09-22T01:01:40.079206+0000","last_fullsized":"2024-09-22T01:01:40.079206+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":11629715,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":15,"num_read":827,"num_read_kb":6095,"num_write":221,"num_write_kb":3775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":62,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.090925+0000","last_change":"2024-09-22T01:01:27.722404+0000","last_active":"2024-09-22T01:01:40.090925+0000","last_peered":"2024-09-22T01:01:40.090925+0000","last_clean":"2024-09-22T01:01:40.090925+0000","last_became_active":"2024-09-22T01:01:27.721384+0000","last_became_peered":"2024-09-22T01:01:27.721384+0000","last_unstale":"2024-09-22T01:01:40.090925+0000","last_undegraded":"2024-09-22T01:01:40.090925+0000","last_fullsized":"2024-09-22T01:01:40.090925+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:10.574832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"63'245","reported_seq":1052,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.077344+0000","last_change":"2024-09-22T01:00:49.709391+0000","last_active":"2024-09-22T01:01:40.077344+0000","last_peered":"2024-09-22T01:01:40.077344+0000","last_clean":"2024-09-22T01:01:40.077344+0000","last_became_active":"2024-09-22T01:00:47.714544+0000","last_became_peered":"2024-09-22T01:00:47.714544+0000","last_unstale":"2024-09-22T01:01:40.077344+0000","last_undegraded":"2024-09-22T01:01:40.077344+0000","last_fullsized":"2024-09-22T01:01:40.077344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:22:01.711751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":19266879,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":1638,"num_read_kb":7421,"num_write":294,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.081738+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:01:40.081738+0000","last_peered":"2024-09-22T01:01:40.081738+0000","last_clean":"2024-09-22T01:01:40.081738+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:01:40.081738+0000","last_undegraded":"2024-09-22T01:01:40.081738+0000","last_fullsized":"2024-09-22T01:01:40.081738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"63'168","reported_seq":1477,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:39.075289+0000","last_change":"2024-09-22T01:01:40.079175+0000","last_active":"2024-09-22T01:01:39.075289+0000","last_peered":"2024-09-22T01:01:39.051152+0000","last_clean":"2024-09-22T01:01:39.051152+0000","last_became_active":"2024-09-22T01:00:47.734922+0000","last_became_peered":"2024-09-22T01:00:47.734922+0000","last_unstale":"2024-09-22T01:01:39.075289+0000","last_undegraded":"2024-09-22T01:01:39.075289+0000","last_fullsized":"2024-09-22T01:01:39.075289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":10281492,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":1530,"num_read_kb":6497,"num_write":196,"num_write_kb":7790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.086315+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:01:40.086315+0000","last_peered":"2024-09-22T01:01:40.086315+0000","last_clean":"2024-09-22T01:01:40.086315+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:01:40.086315+0000","last_undegraded":"2024-09-22T01:01:40.086315+0000","last_fullsized":"2024-09-22T01:01:40.086315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'150","reported_seq":1056,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.082159+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:01:40.082159+0000","last_peered":"2024-09-22T01:01:40.082159+0000","last_clean":"2024-09-22T01:01:40.082159+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:01:40.082159+0000","last_undegraded":"2024-09-22T01:01:40.082159+0000","last_fullsized":"2024-09-22T01:01:40.082159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:01:07.680450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":14809543,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":1025,"num_read_kb":5164,"num_write":171,"num_write_kb":11043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":61,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.090715+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:01:40.090715+0000","last_peered":"2024-09-22T01:01:40.090715+0000","last_clean":"2024-09-22T01:01:40.090715+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:01:40.090715+0000","last_undegraded":"2024-09-22T01:01:40.090715+0000","last_fullsized":"2024-09-22T01:01:40.090715+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":258,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.086606+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:01:40.086606+0000","last_peered":"2024-09-22T01:01:40.086606+0000","last_clean":"2024-09-22T01:01:40.086606+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:01:40.086606+0000","last_undegraded":"2024-09-22T01:01:40.086606+0000","last_fullsized":"2024-09-22T01:01:40.086606+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"63'156","reported_seq":2537,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.081693+0000","last_change":"2024-09-22T01:00:49.735787+0000","last_active":"2024-09-22T01:01:40.081693+0000","last_peered":"2024-09-22T01:01:40.081693+0000","last_clean":"2024-09-22T01:01:40.081693+0000","last_became_active":"2024-09-22T01:00:47.720325+0000","last_became_peered":"2024-09-22T01:00:47.720325+0000","last_unstale":"2024-09-22T01:01:40.081693+0000","last_undegraded":"2024-09-22T01:01:40.081693+0000","last_fullsized":"2024-09-22T01:01:40.081693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:13:05.423429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":18644199,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":3359,"num_read_kb":5929,"num_write":127,"num_write_kb":1398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"63'194","reported_seq":658,"reported_epoch":65,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:40.081427+0000","last_change":"2024-09-22T01:01:40.081427+0000","last_active":"2024-09-22T01:01:39.076682+0000","last_peered":"2024-09-22T01:01:38.710103+0000","last_clean":"2024-09-22T01:01:38.710103+0000","last_became_active":"2024-09-22T01:00:47.727151+0000","last_became_peered":"2024-09-22T01:00:47.727151+0000","last_unstale":"2024-09-22T01:01:40.081427+0000","last_undegraded":"2024-09-22T01:01:40.081427+0000","last_fullsized":"2024-09-22T01:01:40.081427+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":21342257,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":986,"num_read_kb":1324,"num_write":256,"num_write_kb":3853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"66'260","reported_seq":1666,"reported_epoch":66,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:01:41.637239+0000","last_change":"2024-09-22T01:01:41.448353+0000","last_active":"2024-09-22T01:01:41.637239+0000","last_peered":"2024-09-22T01:01:41.637239+0000","last_clean":"2024-09-22T01:01:38.542515+0000","last_became_active":"2024-09-22T01:01:41.108349+0000","last_became_peered":"2024-09-22T01:01:41.108349+0000","last_unstale":"2024-09-22T01:01:41.637239+0000","last_undegraded":"2024-09-22T01:01:41.637239+0000","last_fullsized":"2024-09-22T01:01:41.087837+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":13407088,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":2336,"num_read_kb":3265,"num_write":306,"num_write_kb":3847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":167229,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6],"avail_no_missing":["6","1","4"],"object_location_counts":[{"shards":"6","objects":27}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"63'272","reported_seq":1483,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.082193+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:01:40.082193+0000","last_peered":"2024-09-22T01:01:40.082193+0000","last_clean":"2024-09-22T01:01:40.082193+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:01:40.082193+0000","last_undegraded":"2024-09-22T01:01:40.082193+0000","last_fullsized":"2024-09-22T01:01:40.082193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:33.834395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":6978197,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":2917,"num_read_kb":13913,"num_write":348,"num_write_kb":6301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"63'128","reported_seq":340,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.091742+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:01:40.091742+0000","last_peered":"2024-09-22T01:01:40.091742+0000","last_clean":"2024-09-22T01:01:40.091742+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:01:40.091742+0000","last_undegraded":"2024-09-22T01:01:40.091742+0000","last_fullsized":"2024-09-22T01:01:40.091742+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:41.121057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20821817,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":2071,"num_read_kb":9890,"num_write":119,"num_write_kb":3154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"63'94","reported_seq":2458,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.086683+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:01:40.086683+0000","last_peered":"2024-09-22T01:01:40.086683+0000","last_clean":"2024-09-22T01:01:40.086683+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:01:40.086683+0000","last_undegraded":"2024-09-22T01:01:40.086683+0000","last_fullsized":"2024-09-22T01:01:40.086683+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:11.501024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":23153032,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":2216,"num_read_kb":18360,"num_write":93,"num_write_kb":12052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"63'179","reported_seq":1908,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:39.080077+0000","last_change":"2024-09-22T01:01:40.088607+0000","last_active":"2024-09-22T01:01:39.080077+0000","last_peered":"2024-09-22T01:01:39.055192+0000","last_clean":"2024-09-22T01:01:39.055192+0000","last_became_active":"2024-09-22T01:00:47.732705+0000","last_became_peered":"2024-09-22T01:00:47.732705+0000","last_unstale":"2024-09-22T01:01:39.080077+0000","last_undegraded":"2024-09-22T01:01:39.080077+0000","last_fullsized":"2024-09-22T01:01:39.080077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":36970609,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":1634,"num_read_kb":9741,"num_write":184,"num_write_kb":11227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'107","reported_seq":793,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:39.084890+0000","last_change":"2024-09-22T01:01:40.088802+0000","last_active":"2024-09-22T01:01:39.084890+0000","last_peered":"2024-09-22T01:01:38.978965+0000","last_clean":"2024-09-22T01:01:38.978965+0000","last_became_active":"2024-09-22T01:00:47.732593+0000","last_became_peered":"2024-09-22T01:00:47.732593+0000","last_unstale":"2024-09-22T01:01:39.084890+0000","last_undegraded":"2024-09-22T01:01:39.084890+0000","last_fullsized":"2024-09-22T01:01:39.084890+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":9477662,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":1344,"num_read_kb":2111,"num_write":99,"num_write_kb":6670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"66'158","reported_seq":2149,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-22T01:01:41.441319+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:01:41.441319+0000","last_peered":"2024-09-22T01:01:41.441319+0000","last_clean":"2024-09-22T01:01:41.441319+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:01:41.441319+0000","last_undegraded":"2024-09-22T01:01:41.441319+0000","last_fullsized":"2024-09-22T01:01:41.441319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:07:46.125206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":25920599,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":2461,"num_read_kb":9078,"num_write":131,"num_write_kb":7473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"62'214","reported_seq":795,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:39.079950+0000","last_change":"2024-09-22T01:01:40.088907+0000","last_active":"2024-09-22T01:01:39.079950+0000","last_peered":"2024-09-22T01:01:38.081642+0000","last_clean":"2024-09-22T01:01:38.081642+0000","last_became_active":"2024-09-22T01:00:47.733104+0000","last_became_peered":"2024-09-22T01:00:47.733104+0000","last_unstale":"2024-09-22T01:01:39.079950+0000","last_undegraded":"2024-09-22T01:01:39.079950+0000","last_fullsized":"2024-09-22T01:01:39.079950+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":15555596,"num_objects":29,"num_object_clones":11,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":1212,"num_read_kb":3344,"num_write":307,"num_write_kb":1657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"63'102","reported_seq":985,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-22T01:01:40.078603+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:01:40.078603+0000","last_peered":"2024-09-22T01:01:40.078603+0000","last_clean":"2024-09-22T01:01:40.078603+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:01:40.078603+0000","last_undegraded":"2024-09-22T01:01:40.078603+0000","last_fullsized":"2024-09-22T01:01:40.078603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:27:47.191377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":14451594,"num_objects":36,"num_object_clones":13,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":812,"num_read_kb":1192,"num_write":84,"num_write_kb":3504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"63'109","reported_seq":957,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-22T01:01:39.082513+0000","last_change":"2024-09-22T01:01:40.088565+0000","last_active":"2024-09-22T01:01:39.082513+0000","last_peered":"2024-09-22T01:01:39.059747+0000","last_clean":"2024-09-22T01:01:39.059747+0000","last_became_active":"2024-09-22T01:00:47.733697+0000","last_became_peered":"2024-09-22T01:00:47.733697+0000","last_unstale":"2024-09-22T01:01:39.082513+0000","last_undegraded":"2024-09-22T01:01:39.082513+0000","last_fullsized":"2024-09-22T01:01:39.082513+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":13945434,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":791,"num_read_kb":9917,"num_write":177,"num_write_kb":9487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":607685665,"num_objects":945,"num_object_clones":261,"num_object_copies":1890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":394,"num_read":72973,"num_read_kb":274771,"num_write":7487,"num_write_kb":171902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":28373038,"num_keys_recovered":146,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154034176,"data_stored":183292336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":6217,"ondisk_log_size":6217,"up":64,"acting":51,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37728,"kb_used_data":6540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334112,"statfs":{"total":96636764160,"available":96598130688,"internally_reserved":0,"allocated":6696960,"data_stored":7317727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299396,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62224,"kb_used_data":29940,"kb_used_omap":4,"kb_used_meta":32251,"kb_avail":94309616,"statfs":{"total":96636764160,"available":96573046784,"internally_reserved":0,"allocated":30658560,"data_stored":37844153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":33025614},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60068,"kb_used_data":24720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94311772,"statfs":{"total":96636764160,"available":96575254528,"internally_reserved":0,"allocated":25313280,"data_stored":29744828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542162,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50812,"kb_used_data":23780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94321028,"statfs":{"total":96636764160,"available":96584732672,"internally_reserved":0,"allocated":24350720,"data_stored":26366683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40260,"kb_used_data":4908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331580,"statfs":{"total":96636764160,"available":96595537920,"internally_reserved":0,"allocated":5025792,"data_stored":6763659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43196,"kb_used_data":7844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328644,"statfs":{"total":96636764160,"available":96592531456,"internally_reserved":0,"allocated":8032256,"data_stored":10492009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63808,"kb_used_data":24296,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94308032,"statfs":{"total":96636764160,"available":96571424768,"internally_reserved":0,"allocated":24879104,"data_stored":26469496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23535616,"data_stored":25539885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7278592,"data_stored":10152747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4276224,"data_stored":6424416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23601152,"data_stored":26027440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29917184,"data_stored":37511362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":14663680,"data_stored":17885272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23969792,"data_stored":28815217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:01:44.557 INFO:teuthology.orchestra.run.smithi002.stdout:2827 write 0x45dac83 thru 0x45e019e (0x551c bytes) 2024-09-22T01:01:44.580 INFO:teuthology.orchestra.run.smithi002.stdout:2828 trunc from 0x8125655 to 0x9ee3e68 2024-09-22T01:01:44.600 INFO:teuthology.orchestra.run.smithi002.stdout:2829 write 0x9f4d454 thru 0x9f56737 (0x92e4 bytes) 2024-09-22T01:01:44.614 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:44.614 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:44.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:44.636 INFO:teuthology.orchestra.run.smithi002.stdout:2831 writesame 0xb1b3846 thru 0xb1b5afd data_size 0x8(0x22b8 bytes) 2024-09-22T01:01:44.637 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.637 INFO:teuthology.orchestra.run.smithi002.stdout:2835 writesame 0x5b723c9 thru 0x5b7a8f9 data_size 0x1(0x8531 bytes) 2024-09-22T01:01:44.644 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:44.639+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:44.670 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:44.670 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:44.678 INFO:teuthology.orchestra.run.smithi002.stdout:2838 writesame 0xb97d90f thru 0xb989eb9 data_size 0x1(0xc5ab bytes) 2024-09-22T01:01:44.678 INFO:teuthology.orchestra.run.smithi002.stdout:2839 read 0x57c25be thru 0x57c3504 (0xf47 bytes) 2024-09-22T01:01:44.678 INFO:teuthology.orchestra.run.smithi002.stdout:2840 write 0x5bf5625 thru 0x5bfe1bc (0x8b98 bytes) 2024-09-22T01:01:44.691 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:44.691 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:44.700 INFO:teuthology.orchestra.run.smithi002.stdout:2841 trunc from 0xb989eba to 0x3184cc4 2024-09-22T01:01:44.722 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:44.722 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:44.745 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:44.743+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:44.749 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:44.749 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:44.774 INFO:teuthology.orchestra.run.smithi002.stdout:2843 writesame 0x87e121a thru 0x87edd1e data_size 0x1(0xcb05 bytes) 2024-09-22T01:01:44.779 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:44.779 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:44.784 INFO:teuthology.orchestra.run.smithi002.stdout:2844 trunc from 0x87edd1f to 0x1e0ad5 2024-09-22T01:01:44.791 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:44.791 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:44.791 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:44.792 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:44.792 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:44.813 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:44.814 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:44.814 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:44.840 INFO:teuthology.orchestra.run.smithi002.stdout:2845 punch from 0x16f796 to 0x177916, (0x8180 bytes) 2024-09-22T01:01:44.840 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:44.840 INFO:teuthology.orchestra.run.smithi002.stdout:2848 read 0x7a210 thru 0x8564a (0xb43b bytes) 2024-09-22T01:01:44.841 INFO:teuthology.orchestra.run.smithi002.stdout:2849 clone 26 order 18 su 1024 sc 8 2024-09-22T01:01:44.842 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:44.842 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:44.845 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:44.843+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:44.860 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:44.860 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:44.932 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38846 3.1 3:823bb6bd:::rbd_data.117c55693b3.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.444951043, 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049688678999999999, 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:44.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38846 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487020+0000", 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.023e-06 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497105+0000", 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010085393 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497147+0000", 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1866999999999997e-05 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497210+0000", 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2324999999999997e-05 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497457+0000", 2024-09-22T01:01:44.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024715099999999998 2024-09-22T01:01:44.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497726+0000", 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026944000000000002 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.498317+0000", 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059106199999999995 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.516775+0000", 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018457952 2024-09-22T01:01:44.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536527+0000", 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019751484 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536622+0000", 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5529e-05 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536704+0000", 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1452999999999994e-05 2024-09-22T01:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55415 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 503808~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.497733+0000", 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4342334660000002, 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048933005000000002, 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55415 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497733+0000", 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497733+0000", 2024-09-22T01:01:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497733+0000", 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1199999999999999e-07 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497734+0000", 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.018e-06 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497737+0000", 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.022e-06 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497739+0000", 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4940000000000002e-06 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544791+0000", 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047051608000000002 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5760999999999993e-05 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546666+0000", 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00177889 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55527 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 18432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.944 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.497742+0000", 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.434223604, 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047598541000000001, 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55527 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497742+0000", 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497742+0000", 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497743+0000", 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1299999999999999e-07 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497744+0000", 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0610000000000001e-06 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497747+0000", 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497750+0000", 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5160000000000001e-06 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545103+0000", 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047353108999999997 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545196+0000", 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3323000000000001e-05 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545341+0000", 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145029 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55528 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.497753+0000", 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4342130850000001, 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047967865999999998, 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55528 2024-09-22T01:01:44.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497753+0000", 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497753+0000", 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498606+0000", 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00085334300000000001 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498610+0000", 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7629999999999998e-06 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498623+0000", 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2632999999999999e-05 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498631+0000", 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0180000000000001e-06 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545371+0000", 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046739979000000001 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545459+0000", 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8158000000000005e-05 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545721+0000", 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026197200000000002 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55529 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 75776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498663+0000", 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.433302962, 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047357508999999999, 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55529 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498663+0000", 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498663+0000", 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498664+0000", 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498675+0000", 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1565e-05 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498685+0000", 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0139999999999999e-05 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498692+0000", 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545767+0000", 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047074963999999997 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545880+0000", 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112509 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546021+0000", 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000141116 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55530 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498725+0000", 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4332409660000001, 2024-09-22T01:01:44.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047552522, 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55530 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498725+0000", 2024-09-22T01:01:44.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498725+0000", 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498726+0000", 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498729+0000", 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.162e-06 2024-09-22T01:01:44.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498737+0000", 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1589999999999993e-06 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498743+0000", 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8610000000000003e-06 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546047+0000", 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047303975999999998 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546134+0000", 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7113000000000004e-05 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546278+0000", 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014368200000000001 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55531 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 133120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498752+0000", 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4332143099999999, 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047885688000000003, 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55531 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498752+0000", 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498752+0000", 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498752+0000", 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3000000000000001e-07 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498755+0000", 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3609999999999999e-06 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498762+0000", 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8730000000000001e-06 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498767+0000", 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4269999999999999e-06 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047542327000000002 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546397+0000", 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7410999999999994e-05 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546637+0000", 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000240759 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55532 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498777+0000", 2024-09-22T01:01:44.966 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.433188635, 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048067331999999997, 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55532 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498777+0000", 2024-09-22T01:01:44.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498777+0000", 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498778+0000", 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498781+0000", 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.599e-06 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498788+0000", 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3590000000000002e-06 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498793+0000", 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4260000000000002e-06 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546716+0000", 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047922431000000001 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546781+0000", 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5140000000000003e-05 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546845+0000", 2024-09-22T01:01:44.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3895999999999999e-05 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55533 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498813+0000", 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.43315306, 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048206710999999999, 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55533 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498813+0000", 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498813+0000", 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498814+0000", 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498837+0000", 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3594000000000001e-05 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498847+0000", 2024-09-22T01:01:44.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8560000000000002e-06 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498852+0000", 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546870+0000", 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048017277999999997 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546921+0000", 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1845000000000003e-05 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547020+0000", 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8290999999999998e-05 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55534 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498882+0000", 2024-09-22T01:01:44.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.433084392, 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048300441999999999, 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55534 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498882+0000", 2024-09-22T01:01:44.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498882+0000", 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498882+0000", 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7100000000000001e-07 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498885+0000", 2024-09-22T01:01:44.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8009999999999999e-06 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498893+0000", 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7989999999999994e-06 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498898+0000", 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2680000000000002e-06 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547046+0000", 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048147476000000002 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547095+0000", 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9611000000000003e-05 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547182+0000", 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6716000000000001e-05 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55535 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498906+0000", 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4330601839999999, 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048432527000000003, 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55535 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498906+0000", 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498906+0000", 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498906+0000", 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:01:44.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498909+0000", 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6079999999999998e-06 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498915+0000", 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498920+0000", 2024-09-22T01:01:44.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7249999999999997e-06 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547208+0000", 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048287494 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547259+0000", 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1632999999999998e-05 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547338+0000", 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9235999999999998e-05 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55536 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 276480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498927+0000", 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.433039371, 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049496845999999997, 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55536 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498927+0000", 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498927+0000", 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498927+0000", 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2200000000000004e-07 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498929+0000", 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8139999999999999e-06 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498935+0000", 2024-09-22T01:01:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8019999999999997e-06 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498939+0000", 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2320000000000003e-06 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547364+0000", 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048425134000000002 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547426+0000", 2024-09-22T01:01:44.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1908000000000004e-05 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548424+0000", 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099743399999999995 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55537 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498981+0000", 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4329847920000001, 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049635472999999999, 2024-09-22T01:01:44.988 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55537 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498981+0000", 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498981+0000", 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498982+0000", 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6499999999999999e-07 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498985+0000", 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.802e-06 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498993+0000", 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4209999999999995e-06 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498999+0000", 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548457+0000", 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049458624 2024-09-22T01:01:44.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548536+0000", 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8374999999999994e-05 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548617+0000", 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1023e-05 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55538 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 333824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499017+0000", 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4329488480000001, 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049965862999999999, 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55538 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499017+0000", 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499017+0000", 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499018+0000", 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:44.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499019+0000", 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8190000000000001e-06 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499024+0000", 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4220000000000002e-06 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499027+0000", 2024-09-22T01:01:44.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1690000000000001e-06 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548700+0000", 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049673292000000001 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548781+0000", 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0359999999999996e-05 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:44.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548983+0000", 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020243799999999999 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4329217280000002, 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:44.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:01:44.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:01:45.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4328938849999999, 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:01:45.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:01:45.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4328616690000002, 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:01:45.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:45.010 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4328399310000002, 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:01:45.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4328202760000002, 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:01:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.432789262, 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.023 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:45.025 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:45.025 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:45.019+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:45.125 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:45.123+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:45.197 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:45.197 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:45.212 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:45.254 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:45.251+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:45.327 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:45.323+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:45.384 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:45.384 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:45.400 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:45.426 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-22T01:01:45.428 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:45.423+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:45.529 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:45.527+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:45.539 INFO:teuthology.orchestra.run.smithi002.stdout:2850 write 0x679b088 thru 0x67a834e (0xd2c7 bytes) 2024-09-22T01:01:45.559 INFO:teuthology.orchestra.run.smithi002.stdout:2851 read 0x56e9b83 thru 0x56f481a (0xac98 bytes) 2024-09-22T01:01:45.559 INFO:teuthology.orchestra.run.smithi002.stdout:2852 read 0x39072c thru 0x394e9f (0x4774 bytes) 2024-09-22T01:01:45.567 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485357956999998, 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485339097999997, 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485320913000002, 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485303125000002, 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485284567000001, 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485266514999999, 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485249166000003, 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485227213000002, 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485209677999997, 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485185891999997, 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485168029999997, 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485140792999999, 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485122087999997, 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485098591000003, 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485074621999999, 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.485055406999997, 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.484050857, 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.481597669000003, 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.481575589000002, 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.481544904000003, 2024-09-22T01:01:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.668 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:45.669 INFO:teuthology.orchestra.run.smithi002.stdout:2853 writesame 0xc9b7c47 thru 0xc9c1fbf data_size 0x1(0xa379 bytes) 2024-09-22T01:01:45.669 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:45.669 INFO:teuthology.orchestra.run.smithi002.stdout:2856 punch from 0x46e04ec to 0x46ebaaf, (0xb5c3 bytes) 2024-09-22T01:01:45.669 INFO:teuthology.orchestra.run.smithi002.stdout:2859 read 0x1bf413e thru 0x1bf85d1 (0x4494 bytes) 2024-09-22T01:01:45.669 INFO:teuthology.orchestra.run.smithi002.stdout:2861 write 0xa95307c thru 0xa95ba1c (0x89a1 bytes) 2024-09-22T01:01:45.671 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:45.671 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:45.671 INFO:teuthology.orchestra.run.smithi002.stdout:2864 writesame 0x9669013 thru 0x966b082 data_size 0x18(0x2070 bytes) 2024-09-22T01:01:45.671 INFO:teuthology.orchestra.run.smithi002.stdout:2865 trunc from 0xc9c1fc0 to 0x21c70b8 2024-09-22T01:01:45.671 INFO:teuthology.orchestra.run.smithi002.stdout:2867 trunc from 0x21c70b8 to 0x27da679 2024-09-22T01:01:45.672 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:45.667+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:45.706 INFO:teuthology.orchestra.run.smithi002.stdout:2868 write 0x7641187 thru 0x7647e6a (0x6ce4 bytes) 2024-09-22T01:01:45.734 INFO:teuthology.orchestra.run.smithi002.stdout:2869 trunc from 0x7647e6b to 0x1f31be5 2024-09-22T01:01:45.759 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:45.770 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:45.767+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:45.797 INFO:teuthology.orchestra.run.smithi002.stdout:2873 write 0xd12905f thru 0xd12ba1f (0x29c1 bytes) 2024-09-22T01:01:45.812 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:59067 3.0 e70/22)", 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.812729+0000", 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.00024762299999999998, 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026929299999999998, 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812729+0000", 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812729+0000", 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812732+0000", 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.446e-06 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812750+0000", 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8315999999999999e-05 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812795+0000", 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4781999999999999e-05 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812817+0000", 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1846000000000001e-05 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:59069 3.1a e70/67)", 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3015000000000003e-05, 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015999499999999999, 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812885+0000", 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-06 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812930+0000", 2024-09-22T01:01:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4731999999999998e-05 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812963+0000", 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3791999999999998e-05 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812986+0000", 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2493e-05 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:01:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 2 2024-09-22T01:01:45.819 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:45.819 INFO:teuthology.orchestra.run.smithi002.stdout:2874 writesame 0x2f644a thru 0x2ff8c6 data_size 0x1(0x947d bytes) 2024-09-22T01:01:45.830 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:45.871 INFO:teuthology.orchestra.run.smithi002.stdout:2875 trunc from 0xd12ba20 to 0x3f2d462 2024-09-22T01:01:45.872 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:45.867+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:45.926 INFO:teuthology.orchestra.run.smithi002.stdout:2876 write 0xe5b85f0 thru 0xe5bddad (0x57be bytes) 2024-09-22T01:01:45.942 INFO:teuthology.orchestra.run.smithi002.stdout:2877 write 0x9e119b7 thru 0x9e1de7c (0xc4c6 bytes) 2024-09-22T01:01:45.956 INFO:teuthology.orchestra.run.smithi002.stdout:2879 writesame 0xa290ec8 thru 0xa2980e8 data_size 0x1(0x7221 bytes) 2024-09-22T01:01:45.972 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:45.967+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:45.981 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:45.981 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:45.981 INFO:teuthology.orchestra.run.smithi002.stdout:2884 read 0x861d8f2 thru 0x8627be1 (0xa2f0 bytes) 2024-09-22T01:01:45.981 INFO:teuthology.orchestra.run.smithi002.stdout:2887 writesame 0xa241836 thru 0xa24b41f data_size 0x7(0x9bea bytes) 2024-09-22T01:01:45.987 INFO:teuthology.orchestra.run.smithi002.stdout:2888 write 0x1f01491 thru 0x1f0f380 (0xdef0 bytes) 2024-09-22T01:01:46.002 INFO:teuthology.orchestra.run.smithi002.stdout:2890 writesame 0x47e17cf thru 0x47ea1b0 data_size 0x6f(0x89e2 bytes) 2024-09-22T01:01:46.017 INFO:teuthology.orchestra.run.smithi002.stdout:2892 writesame 0x92089a5 thru 0x920dfd9 data_size 0x1(0x5635 bytes) 2024-09-22T01:01:46.028 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:46.028 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:46.037 INFO:teuthology.orchestra.run.smithi002.stdout:2893 read 0x18e9c6f thru 0x18f192e (0x7cc0 bytes) 2024-09-22T01:01:46.037 INFO:teuthology.orchestra.run.smithi002.stdout:2894 write 0xb78266a thru 0xb788a8e (0x6425 bytes) 2024-09-22T01:01:46.046 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:46.050 INFO:teuthology.orchestra.run.smithi002.stdout:2895 punch from 0xbbf25b to 0xbbf5f4, (0x399 bytes) 2024-09-22T01:01:46.072 INFO:teuthology.orchestra.run.smithi002.stdout:2897 trunc from 0xe5bddae to 0x9d20d7 2024-09-22T01:01:46.073 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:46.067+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:46.110 INFO:teuthology.orchestra.run.smithi002.stdout:2898 punch from 0x5f75a3 to 0x5fa520, (0x2f7d bytes) 2024-09-22T01:01:46.130 INFO:teuthology.orchestra.run.smithi002.stdout:2899 write 0x624ca0d thru 0x6258e42 (0xc436 bytes) 2024-09-22T01:01:46.144 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:46.155 INFO:teuthology.orchestra.run.smithi002.stdout:2902 write 0x64fbb01 thru 0x65085c1 (0xcac1 bytes) 2024-09-22T01:01:46.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:01:46.170 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:46.191 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8964 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'20, mlcod=32'17)", 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.053572537999997, 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077894110000000004, 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136525+0000", 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3540000000000001e-06 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137650+0000", 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011257190000000001 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137671+0000", 2024-09-22T01:01:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0560999999999998e-05 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137680+0000", 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137763+0000", 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2472999999999995e-05 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137793+0000", 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-05 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144267+0000", 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064739139999999999 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144313+0000", 2024-09-22T01:01:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5624999999999998e-05 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8968 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'21, mlcod=32'17)", 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.049146493000002, 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004894771, 2024-09-22T01:01:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140952+0000", 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1849999999999998e-06 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.141985+0000", 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001032984 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142001+0000", 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6776999999999999e-05 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142010+0000", 2024-09-22T01:01:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4640000000000006e-06 2024-09-22T01:01:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142101+0000", 2024-09-22T01:01:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1312999999999998e-05 2024-09-22T01:01:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142119+0000", 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8202e-05 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145810+0000", 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036909970000000001 2024-09-22T01:01:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145844+0000", 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3849000000000001e-05 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9077 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'31, mlcod=32'29)", 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.794822117999999, 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034796760000000001, 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395277+0000", 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8600000000000001e-06 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395294+0000", 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7513e-05 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395337+0000", 2024-09-22T01:01:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2749e-05 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395358+0000", 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1097000000000001e-05 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395548+0000", 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189829 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395593+0000", 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4919000000000001e-05 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398634+0000", 2024-09-22T01:01:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003041373 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398753+0000", 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119336 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35894 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'145, mlcod=47'143)", 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.037081240999999, 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055540160000000002, 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8700000000000002e-07 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153852+0000", 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083735000000000005 2024-09-22T01:01:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153865+0000", 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2804e-05 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153872+0000", 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:01:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153943+0000", 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0908000000000006e-05 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153968+0000", 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5530000000000001e-05 2024-09-22T01:01:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158535+0000", 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045670559999999999 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158569+0000", 2024-09-22T01:01:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210999999999998e-05 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35895 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'147, mlcod=47'143)", 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.034281085, 2024-09-22T01:01:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031074570000000001, 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155816+0000", 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.666e-06 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155839+0000", 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2768e-05 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155940+0000", 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010056999999999999 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155982+0000", 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1850000000000001e-05 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.156671+0000", 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00068920600000000004 2024-09-22T01:01:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.156712+0000", 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0917999999999997e-05 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158885+0000", 2024-09-22T01:01:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021735859999999999 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158922+0000", 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6893000000000001e-05 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37899 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'73, mlcod=48'71)", 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.010577418, 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012095786000000001, 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179519+0000", 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4099999999999997e-07 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183241+0000", 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037216110000000001 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183261+0000", 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9799000000000001e-05 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183274+0000", 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3046e-05 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183374+0000", 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010009399999999999 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183399+0000", 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5375999999999999e-05 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191540+0000", 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081403029999999998 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191614+0000", 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4715999999999994e-05 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37936 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'74, mlcod=48'72)", 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.840816347000001, 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003111749, 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349282+0000", 2024-09-22T01:01:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2970000000000002e-06 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349299+0000", 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7278999999999998e-05 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349331+0000", 2024-09-22T01:01:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2218000000000002e-05 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349346+0000", 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4856e-05 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349548+0000", 2024-09-22T01:01:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020148599999999999 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349608+0000", 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9893999999999999e-05 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.352260+0000", 2024-09-22T01:01:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002652583 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.352391+0000", 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013113600000000001 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:43527 3.9 e60/22 3:96fe8936:::rbd_data.117c83c8bd76.000000000000001b:head v 60'182, mlcod=60'180)", 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.348526207000001, 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005051725, 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841572+0000", 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845187+0000", 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036148880000000001 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845210+0000", 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2926e-05 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845221+0000", 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.17e-05 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845322+0000", 2024-09-22T01:01:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101003 2024-09-22T01:01:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845348+0000", 2024-09-22T01:01:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5361e-05 2024-09-22T01:01:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846545+0000", 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011971659999999999 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846621+0000", 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6631000000000005e-05 2024-09-22T01:01:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:53802 3.13 e66/65 3:cb3ff6d1:::rbd_data.117c93605fe7.00000000000000ed:head v 66'110, mlcod=0'0)", 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7528353719999998, 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003122391, 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437261+0000", 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4799999999999998e-07 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437266+0000", 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-22T01:01:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437276+0000", 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6420000000000002e-06 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437283+0000", 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9450000000000004e-06 2024-09-22T01:01:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.438702+0000", 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014196409999999999 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.438725+0000", 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2456e-05 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.440350+0000", 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001624955 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.440383+0000", 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3204999999999998e-05 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56176 3.9 3:91eeadac:::rbd_data.117c93605fe7.00000000000001f7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2913419040000003, 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038039409999999999, 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56176 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898756+0000", 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.832e-06 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898763+0000", 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3440000000000003e-06 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898778+0000", 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4982e-05 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898787+0000", 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7479999999999999e-06 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898845+0000", 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8616999999999998e-05 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898903+0000", 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7623e-05 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898982+0000", 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9124999999999999e-05 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.899559+0000", 2024-09-22T01:01:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00057709899999999997 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902402+0000", 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028426990000000002 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902461+0000", 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8607999999999999e-05 2024-09-22T01:01:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902558+0000", 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7263999999999996e-05 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56184 3.9 3:914e0854:::rbd_data.117c93605fe7.000000000000021b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2887334609999996, 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040309569999999999, 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56184 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901364+0000", 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.48e-06 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901369+0000", 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1050000000000003e-06 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901384+0000", 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5098e-05 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901392+0000", 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9969999999999998e-06 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901444+0000", 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2268999999999997e-05 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901496+0000", 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1864999999999999e-05 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901562+0000", 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5718999999999999e-05 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902103+0000", 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054061300000000001 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905243+0000", 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031400059999999999 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905305+0000", 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2527000000000006e-05 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905393+0000", 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8277999999999997e-05 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56364 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [write 36377~4583] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1454373980000003, 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034458259999999999, 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56364 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044660+0000", 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.046e-06 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044672+0000", 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2119999999999999e-05 2024-09-22T01:01:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044691+0000", 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9320000000000001e-05 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044698+0000", 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2420000000000003e-06 2024-09-22T01:01:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044756+0000", 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7547999999999999e-05 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044836+0000", 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0657999999999999e-05 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044966+0000", 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000129682 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.046764+0000", 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001797877 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.047933+0000", 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001169139 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.047999+0000", 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5537000000000006e-05 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.048104+0000", 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010565699999999999 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56402 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1148851439999996, 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004133734, 2024-09-22T01:01:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56402 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075212+0000", 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7060000000000001e-06 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075220+0000", 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2590000000000003e-06 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075235+0000", 2024-09-22T01:01:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.537e-05 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075243+0000", 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1159999999999999e-06 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075314+0000", 2024-09-22T01:01:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0535000000000001e-05 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075399+0000", 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5745e-05 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075512+0000", 2024-09-22T01:01:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112575 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.076225+0000", 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071304799999999998 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079253+0000", 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030281520000000001 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079290+0000", 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7240000000000003e-05 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079344+0000", 2024-09-22T01:01:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3987999999999999e-05 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:58441 3.9 3:961db89f:::rbd_data.117c93605fe7.000000000000027c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.44822501, 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035419259999999999, 2024-09-22T01:01:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 58441 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741873+0000", 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7990000000000001e-06 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741879+0000", 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0249999999999999e-06 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741896+0000", 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6962000000000001e-05 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741904+0000", 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4489999999999999e-06 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741961+0000", 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7206000000000001e-05 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742048+0000", 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7078999999999998e-05 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742156+0000", 2024-09-22T01:01:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010733799999999999 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742877+0000", 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072088000000000002 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745306+0000", 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002429512 2024-09-22T01:01:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745365+0000", 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8962000000000002e-05 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745413+0000", 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7713999999999998e-05 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59024 3.9 3:935ab33d:::rbd_data.117ce35b6780.00000000000001db:head [set-alloc-hint object_size 262144 write_size 262144,write 33792~3072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.47251020999999999, 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048336580000000002, 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59024 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717588+0000", 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8030000000000001e-06 2024-09-22T01:01:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717600+0000", 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2028000000000001e-05 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717626+0000", 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5930000000000001e-05 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717639+0000", 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2296000000000001e-05 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717728+0000", 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9091999999999993e-05 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717864+0000", 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013604200000000001 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.718008+0000", 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014423800000000001 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.719029+0000", 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001020897 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722322+0000", 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032925509999999999 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722370+0000", 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8755000000000003e-05 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722419+0000", 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9026000000000001e-05 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59026 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [set-alloc-hint object_size 262144 write_size 262144,write 32768~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46984111499999998, 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046681709999999996, 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59026 2024-09-22T01:01:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720256+0000", 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3969999999999999e-06 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720267+0000", 2024-09-22T01:01:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699e-05 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720285+0000", 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8561999999999999e-05 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720296+0000", 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0324e-05 2024-09-22T01:01:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720362+0000", 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6156999999999997e-05 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720474+0000", 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111656 2024-09-22T01:01:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720641+0000", 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167875 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.721449+0000", 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00080787999999999997 2024-09-22T01:01:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724779+0000", 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033294090000000002 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724864+0000", 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5535e-05 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724923+0000", 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8677000000000001e-05 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59035 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.45090314500000001, 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056261260000000004, 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59035 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739194+0000", 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7599999999999996e-07 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739199+0000", 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1919999999999996e-06 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739212+0000", 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3484e-05 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739222+0000", 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3279999999999995e-06 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739864+0000", 2024-09-22T01:01:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064251300000000005 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739973+0000", 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010924700000000001 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740123+0000", 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014958199999999999 2024-09-22T01:01:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740800+0000", 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067721600000000002 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744735+0000", 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039345889999999996 2024-09-22T01:01:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744779+0000", 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4196999999999997e-05 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744819+0000", 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0002000000000001e-05 2024-09-22T01:01:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:59069 3.1a e70/67)", 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.37721197299999998, 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033876940000000001, 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812885+0000", 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-06 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812930+0000", 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4731999999999998e-05 2024-09-22T01:01:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812963+0000", 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3791999999999998e-05 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812986+0000", 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2493e-05 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.814703+0000", 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017169799999999999 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.814754+0000", 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1035999999999997e-05 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.816213+0000", 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014592100000000001 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.816272+0000", 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8446999999999998e-05 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59186 3.9 3:973f8b5e:::rbd_data.117ce35b6780.000000000000007d:head [set-alloc-hint object_size 262144 write_size 262144,write 131217~7023] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.19204011300000001, 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039873030000000002, 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59186 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998059+0000", 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998076+0000", 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.732e-05 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998110+0000", 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3785999999999998e-05 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998124+0000", 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4544e-05 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998226+0000", 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101544 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998436+0000", 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021000899999999999 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998697+0000", 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000261406 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.999737+0000", 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010393519999999999 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001927+0000", 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021903420000000001 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001985+0000", 2024-09-22T01:01:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7711999999999999e-05 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.002043+0000", 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8477999999999998e-05 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59243 3.9 3:973f8b5e:::rbd_data.117ce35b6780.000000000000007d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.113357624, 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032113020000000001, 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59243 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076741+0000", 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0769999999999999e-06 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076751+0000", 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3929999999999997e-06 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076778+0000", 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7001000000000001e-05 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076790+0000", 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.183e-05 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076875+0000", 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5158e-05 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076967+0000", 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2660999999999999e-05 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.077078+0000", 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110343 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.078598+0000", 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00152005 2024-09-22T01:01:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079834+0000", 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001236343 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079897+0000", 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2724000000000005e-05 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079950+0000", 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2722000000000001e-05 2024-09-22T01:01:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.330 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:46.330 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:46.187+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:46.330 INFO:teuthology.orchestra.run.smithi002.stdout:2906 write 0xdf40922 thru 0xdf45938 (0x5017 bytes) 2024-09-22T01:01:46.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:46.332 INFO:teuthology.orchestra.run.smithi002.stdout:2907 trunc from 0xdf45939 to 0xa880f28 2024-09-22T01:01:46.332 INFO:teuthology.orchestra.run.smithi002.stdout:2909 writesame 0xe5f88f9 thru 0xe605ae7 data_size 0x1(0xd1ef bytes) 2024-09-22T01:01:46.332 INFO:teuthology.orchestra.run.smithi002.stdout:2910 write 0x3941956 thru 0x3949c28 (0x82d3 bytes) 2024-09-22T01:01:46.333 INFO:teuthology.orchestra.run.smithi002.stdout:2911 read 0xc2093e8 thru 0xc2153d4 (0xbfed bytes) 2024-09-22T01:01:46.333 INFO:teuthology.orchestra.run.smithi002.stdout:2913 read 0x36ca40f thru 0x36cf93e (0x5530 bytes) 2024-09-22T01:01:46.333 INFO:teuthology.orchestra.run.smithi002.stdout:2914 read 0xd767584 thru 0xd774f3b (0xd9b8 bytes) 2024-09-22T01:01:46.333 INFO:teuthology.orchestra.run.smithi002.stdout:2915 punch from 0x7850e43 to 0x7860cf8, (0xfeb5 bytes) 2024-09-22T01:01:46.333 INFO:teuthology.orchestra.run.smithi002.stdout:2917 clone 27 order 23 su 2097152 sc 6 2024-09-22T01:01:46.334 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:46.331+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:46.435 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:46.431+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:46.501 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:46.501 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:46.515 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:46.536 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:46.535+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:46.633 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:46.634 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:46.636 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:46.635+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:46.648 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:46.737 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:46.735+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:46.778 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3023051319999999, 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:01:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:01:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3022843030000004, 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:01:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:01:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3022557880000001, 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3022331380000001, 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:01:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3022111430000001, 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3021860790000002, 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:01:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3021636460000003, 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:01:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:01:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3021421689999997, 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:01:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:01:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:01:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3021134109999997, 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:01:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:01:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:01:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3013004190000004, 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:01:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3012757339999999, 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:01:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:01:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2969542000000001, 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:01:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2969253490000003, 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:01:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2969036239999996, 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2968831940000003, 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2968608780000004, 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2967930829999998, 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2967599459999999, 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:01:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:01:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2967360240000003, 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:01:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2967124270000001, 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:46.867 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:46.868 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:46.869 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:46.867+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:46.876 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:46.969 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:46.967+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:47.000 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:47.001 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:47.001 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:47.001 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:47.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:47.032 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:47.049 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:47.049 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:47.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:47.066 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:47.070 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:47.066+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:47.083 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:47.101 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:47.101 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:47.119 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:47.138 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:47.138 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:47.151 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:01:47.159 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:47.171 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:47.166+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:47.178 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:47.178 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:47.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:47.249 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone26 from 0xe605ae8 (overlap 0x1e0ad5) to 0x1808aa 2024-09-22T01:01:47.271 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:47.266+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:47.328 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:47.328 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:47.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:47.372 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:47.366+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:47.467 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-22T01:01:47.471 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:47.471 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:47.473 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:47.470+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:47.486 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:47.498 INFO:teuthology.orchestra.run.smithi002.stdout:2920 trunc from 0xe605ae8 to 0x1e3450b 2024-09-22T01:01:47.574 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:47.570+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:47.597 INFO:teuthology.orchestra.run.smithi002.stdout:2921 write 0x2301c95 thru 0x2302534 (0x8a0 bytes) 2024-09-22T01:01:47.612 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:47.612 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:47.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:47.632 INFO:teuthology.orchestra.run.smithi002.stdout:2923 write 0x9be14cf thru 0x9be5544 (0x4076 bytes) 2024-09-22T01:01:47.639 INFO:teuthology.orchestra.run.smithi002.stdout:2925 write 0x603bf94 thru 0x603cb27 (0xb94 bytes) 2024-09-22T01:01:47.847 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:47.848 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:47.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:2926 write 0xbd7f578 thru 0xbd8be87 (0xc910 bytes) 2024-09-22T01:01:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:2928 write 0x3110507 thru 0x3116331 (0x5e2b bytes) 2024-09-22T01:01:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:2929 write 0x903df97 thru 0x903e875 (0x8df bytes) 2024-09-22T01:01:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:2930 read 0x57f024 thru 0x581e98 (0x2e75 bytes) 2024-09-22T01:01:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:2932 writesame 0xb6cd0d0 thru 0xb6d629b data_size 0x7c(0x91cc bytes) 2024-09-22T01:01:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:2933 write 0x4675a1 thru 0x46d14f (0x5baf bytes) 2024-09-22T01:01:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:2934 write 0x888d70 thru 0x8937a6 (0xaa37 bytes) 2024-09-22T01:01:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:2940 write 0x88902a9 thru 0x889a6de (0xa436 bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2941 writesame 0xd154c66 thru 0xd1568ef data_size 0xd(0x1c8a bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2942 trunc from 0xd1568f0 to 0xa03b2b3 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2945 trunc from 0xa03b2b3 to 0x71175b9 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2947 write 0x23ebb09 thru 0x23f773b (0xbc33 bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2949 read 0x2b63da6 thru 0x2b67f4a (0x41a5 bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2951 punch from 0x701a0d2 to 0x701f8a0, (0x57ce bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2952 write 0x56b0b10 thru 0x56b60ad (0x559e bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2953 write 0xfb6029 thru 0xfbd2c2 (0x729a bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2955 punch from 0x3921c55 to 0x392c029, (0xa3d4 bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2956 read 0x167f5b2 thru 0x168a265 (0xacb4 bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2957 write 0x4abd7a2 thru 0x4ac3289 (0x5ae8 bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:2958 write 0x9241c35 thru 0x9243943 (0x1d0f bytes) 2024-09-22T01:01:47.851 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:2963 write 0x5d230a0 thru 0x5d24b7b (0x1adc bytes) 2024-09-22T01:01:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:2964 punch from 0xf1c2c9 to 0xf267d1, (0xa508 bytes) 2024-09-22T01:01:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:2967 read 0x5cafe84 thru 0x5cbdc76 (0xddf3 bytes) 2024-09-22T01:01:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:2971 write 0xc4744de thru 0xc47a2f4 (0x5e17 bytes) 2024-09-22T01:01:47.852 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:47.842+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:2972 read 0x4a987c thru 0x4ac36f (0x2af4 bytes) 2024-09-22T01:01:47.879 INFO:teuthology.orchestra.run.smithi002.stdout:2973 trunc from 0xc47a2f5 to 0x1ea7268 2024-09-22T01:01:47.924 INFO:teuthology.orchestra.run.smithi002.stdout:2974 write 0x567c75c thru 0x5682889 (0x612e bytes) 2024-09-22T01:01:47.948 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:47.943+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:47.954 INFO:teuthology.orchestra.run.smithi002.stdout:2975 writesame 0x831258f thru 0x83146c6 data_size 0x4(0x2138 bytes) 2024-09-22T01:01:47.954 INFO:teuthology.orchestra.run.smithi002.stdout:2977 write 0x2e482d8 thru 0x2e4dc8b (0x59b4 bytes) 2024-09-22T01:01:47.971 INFO:teuthology.orchestra.run.smithi002.stdout:2980 write 0x851ab6d thru 0x851fb64 (0x4ff8 bytes) 2024-09-22T01:01:47.978 INFO:teuthology.orchestra.run.smithi002.stdout:2982 writesame 0xade890 thru 0xaeb0cb data_size 0x16(0xc83c bytes) 2024-09-22T01:01:47.983 INFO:teuthology.orchestra.run.smithi002.stdout:2983 read 0x6b86702 thru 0x6b86f66 (0x865 bytes) 2024-09-22T01:01:47.983 INFO:teuthology.orchestra.run.smithi002.stdout:2984 write 0x4e5a7ef thru 0x4e64fd3 (0xa7e5 bytes) 2024-09-22T01:01:47.984 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:47.984 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:47.991 INFO:teuthology.orchestra.run.smithi002.stdout:2985 write 0x6e610b1 thru 0x6e69ab3 (0x8a03 bytes) 2024-09-22T01:01:47.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:48.000 INFO:teuthology.orchestra.run.smithi002.stdout:2986 read 0x13e843e thru 0x13f6543 (0xe106 bytes) 2024-09-22T01:01:48.000 INFO:teuthology.orchestra.run.smithi002.stdout:2987 punch from 0x444eaa2 to 0x44552e9, (0x6847 bytes) 2024-09-22T01:01:48.000 INFO:teuthology.orchestra.run.smithi002.stdout:2988 read 0x7278975 thru 0x728294a (0x9fd6 bytes) 2024-09-22T01:01:48.000 INFO:teuthology.orchestra.run.smithi002.stdout:2990 write 0xa3b5a8 thru 0xa4a7e7 (0xf240 bytes) 2024-09-22T01:01:48.020 INFO:teuthology.orchestra.run.smithi002.stdout:2995 write 0xab86379 thru 0xab8f39a (0x9022 bytes) 2024-09-22T01:01:48.027 INFO:teuthology.orchestra.run.smithi002.stdout:2997 punch from 0x8fcbb9a to 0x8fd8d4f, (0xd1b5 bytes) 2024-09-22T01:01:48.027 INFO:teuthology.orchestra.run.smithi002.stdout:2998 punch from 0x201ac49 to 0x201b34d, (0x704 bytes) 2024-09-22T01:01:48.037 INFO:teuthology.orchestra.run.smithi002.stdout:3000 trunc from 0xab8f39b to 0x6f995e9 2024-09-22T01:01:48.048 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:48.042+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:48.059 INFO:teuthology.orchestra.run.smithi002.stdout:3001 read 0x4ac6c8f thru 0x4ac80d6 (0x1448 bytes) 2024-09-22T01:01:48.068 INFO:teuthology.orchestra.run.smithi002.stdout:3002 trunc from 0x6f995e9 to 0xa307267 2024-09-22T01:01:48.079 INFO:teuthology.orchestra.run.smithi002.stdout:3003 writesame 0x9ef060 thru 0x9ef63e data_size 0x3(0x5df bytes) 2024-09-22T01:01:48.082 INFO:teuthology.orchestra.run.smithi002.stdout:3004 write 0x37b4e29 thru 0x37b6cb1 (0x1e89 bytes) 2024-09-22T01:01:48.092 INFO:teuthology.orchestra.run.smithi002.stdout:3006 trunc from 0xa307267 to 0xcb43851 2024-09-22T01:01:48.101 INFO:teuthology.orchestra.run.smithi002.stdout:3007 read 0xa663c8e thru 0xa6719e8 (0xdd5b bytes) 2024-09-22T01:01:48.102 INFO:teuthology.orchestra.run.smithi002.stdout:3008 write 0xfc9dd thru 0x10aa6f (0xe093 bytes) 2024-09-22T01:01:48.107 INFO:teuthology.orchestra.run.smithi002.stdout:3009 writesame 0x1ecc7 thru 0x201bc data_size 0x1(0x14f6 bytes) 2024-09-22T01:01:48.115 INFO:teuthology.orchestra.run.smithi002.stdout:3013 read 0x24a0c20 thru 0x24a74e5 (0x68c6 bytes) 2024-09-22T01:01:48.116 INFO:teuthology.orchestra.run.smithi002.stdout:3014 read 0x683f9e2 thru 0x6843cdd (0x42fc bytes) 2024-09-22T01:01:48.116 INFO:teuthology.orchestra.run.smithi002.stdout:3015 punch from 0x2dea6db to 0x2df8253, (0xdb78 bytes) 2024-09-22T01:01:48.116 INFO:teuthology.orchestra.run.smithi002.stdout:3016 punch from 0x3dea797 to 0x3deefa2, (0x480b bytes) 2024-09-22T01:01:48.116 INFO:teuthology.orchestra.run.smithi002.stdout:3017 read 0x4a7b1a3 thru 0x4a878e9 (0xc747 bytes) 2024-09-22T01:01:48.117 INFO:teuthology.orchestra.run.smithi002.stdout:3018 write 0xdeda9c thru 0xdf5c84 (0x81e9 bytes) 2024-09-22T01:01:48.121 INFO:teuthology.orchestra.run.smithi002.stdout:3019 write 0x8ade72c thru 0x8ae44e2 (0x5db7 bytes) 2024-09-22T01:01:48.128 INFO:teuthology.orchestra.run.smithi002.stdout:3020 write 0x5af7447 thru 0x5b03381 (0xbf3b bytes) 2024-09-22T01:01:48.132 INFO:teuthology.orchestra.run.smithi002.stdout:3021 writesame 0x850dc49 thru 0x85183e9 data_size 0x1(0xa7a1 bytes) 2024-09-22T01:01:48.149 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:48.147+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:48.173 INFO:teuthology.orchestra.run.smithi002.stdout:3022 writesame 0x289d916 thru 0x28abf69 data_size 0x2(0xe654 bytes) 2024-09-22T01:01:48.181 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:48.181 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:48.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:48.208 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:48.208 INFO:teuthology.orchestra.run.smithi002.stdout:3027 read 0x5323b89 thru 0x5328f86 (0x53fe bytes) 2024-09-22T01:01:48.208 INFO:teuthology.orchestra.run.smithi002.stdout:3028 read 0x22400db thru 0x2248f17 (0x8e3d bytes) 2024-09-22T01:01:48.209 INFO:teuthology.orchestra.run.smithi002.stdout:3029 read 0x943a2b0 thru 0x943cf2e (0x2c7f bytes) 2024-09-22T01:01:48.215 INFO:teuthology.orchestra.run.smithi002.stdout:3030 trunc from 0xcb43851 to 0xebd14db 2024-09-22T01:01:48.250 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:48.246+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:48.265 INFO:teuthology.orchestra.run.smithi002.stdout:3031 trunc from 0xebd14db to 0xe517ef 2024-09-22T01:01:48.291 INFO:teuthology.orchestra.run.smithi002.stdout:3034 read 0x702759 thru 0x70321a (0xac2 bytes) 2024-09-22T01:01:48.325 INFO:teuthology.orchestra.run.smithi002.stdout:3035 write 0x87f2d19 thru 0x87f33d7 (0x6bf bytes) 2024-09-22T01:01:48.351 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:48.347+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:48.359 INFO:teuthology.orchestra.run.smithi002.stdout:3037 write 0xd0c49cf thru 0xd0d3d35 (0xf367 bytes) 2024-09-22T01:01:48.376 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:48.376 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:48.390 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:48.397 INFO:teuthology.orchestra.run.smithi002.stdout:3038 trunc from 0xd0d3d36 to 0x22e6cd1 2024-09-22T01:01:48.429 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:48.442 INFO:teuthology.orchestra.run.smithi002.stdout:3040 writesame 0x263bfaf thru 0x2646342 data_size 0x4c(0xa394 bytes) 2024-09-22T01:01:48.445 INFO:teuthology.orchestra.run.smithi002.stdout:3041 punch from 0x1ded3c4 to 0x1ded766, (0x3a2 bytes) 2024-09-22T01:01:48.445 INFO:teuthology.orchestra.run.smithi002.stdout:3042 writesame 0x1b10cd1 thru 0x1b1422f data_size 0x1(0x355f bytes) 2024-09-22T01:01:48.468 INFO:teuthology.orchestra.run.smithi002.stdout:3043 trunc from 0x2646343 to 0x905367 2024-09-22T01:01:48.480 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:48.479+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:48.526 INFO:teuthology.orchestra.run.smithi002.stdout:3045 write 0x9561dce thru 0x956945e (0x7691 bytes) 2024-09-22T01:01:48.533 INFO:teuthology.orchestra.run.smithi002.stdout:3046 punch from 0x593b5c1 to 0x593ef2a, (0x3969 bytes) 2024-09-22T01:01:48.552 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:48.551+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:48.555 INFO:teuthology.orchestra.run.smithi002.stdout:3048 write 0xccba45d thru 0xccbd31d (0x2ec1 bytes) 2024-09-22T01:01:48.562 INFO:teuthology.orchestra.run.smithi002.stdout:3049 punch from 0x16c2532 to 0x16c46c7, (0x2195 bytes) 2024-09-22T01:01:48.562 INFO:teuthology.orchestra.run.smithi002.stdout:3050 read 0xbf9b80c thru 0xbfa2564 (0x6d59 bytes) 2024-09-22T01:01:48.562 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:48.565 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:48.569 INFO:teuthology.orchestra.run.smithi002.stdout:3052 trunc from 0xccbd31e to 0xa76d159 2024-09-22T01:01:48.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:01:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:3054 read 0x6545948 thru 0x6553825 (0xdede bytes) 2024-09-22T01:01:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:3055 write 0x87217d7 thru 0x872d625 (0xbe4f bytes) 2024-09-22T01:01:48.604 INFO:teuthology.orchestra.run.smithi002.stdout:3056 read 0xa3783fc thru 0xa384427 (0xc02c bytes) 2024-09-22T01:01:48.604 INFO:teuthology.orchestra.run.smithi002.stdout:3057 read 0x4e3e03 thru 0x4eca8b (0x8c89 bytes) 2024-09-22T01:01:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:3058 writesame 0x184c91c thru 0x184ea63 data_size 0x18(0x2148 bytes) 2024-09-22T01:01:48.609 INFO:teuthology.orchestra.run.smithi002.stdout:3063 write 0x2d21a61 thru 0x2d29a24 (0x7fc4 bytes) 2024-09-22T01:01:48.613 INFO:teuthology.orchestra.run.smithi002.stdout:3064 punch from 0x5d6428b to 0x5d6a7c1, (0x6536 bytes) 2024-09-22T01:01:48.640 INFO:teuthology.orchestra.run.smithi002.stdout:3066 trunc from 0xa76d159 to 0x11b421 2024-09-22T01:01:48.653 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:48.646+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:48.710 INFO:teuthology.orchestra.run.smithi002.stdout:3067 trunc from 0x11b421 to 0xc4726fb 2024-09-22T01:01:48.723 INFO:teuthology.orchestra.run.smithi002.stdout:3068 write 0x91b3316 thru 0x91c1a71 (0xe75c bytes) 2024-09-22T01:01:48.730 INFO:teuthology.orchestra.run.smithi002.stdout:3069 read 0xae53b86 thru 0xae5de52 (0xa2cd bytes) 2024-09-22T01:01:48.730 INFO:teuthology.orchestra.run.smithi002.stdout:3070 read 0x3425658 thru 0x342c1e0 (0x6b89 bytes) 2024-09-22T01:01:48.730 INFO:teuthology.orchestra.run.smithi002.stdout:3071 read 0x8cde88 thru 0x8d4ab3 (0x6c2c bytes) 2024-09-22T01:01:48.730 INFO:teuthology.orchestra.run.smithi002.stdout:3072 write 0x433a25a thru 0x434348a (0x9231 bytes) 2024-09-22T01:01:48.752 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:48.752 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:48.753 INFO:teuthology.orchestra.run.smithi002.stdout:3073 trunc from 0xc4726fb to 0x6f5c735 2024-09-22T01:01:48.753 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:48.751+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:48.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:01:48.774 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:48.787 INFO:teuthology.orchestra.run.smithi002.stdout:3075 trunc from 0x6f5c735 to 0x1e7da0a 2024-09-22T01:01:48.812 INFO:teuthology.orchestra.run.smithi002.stdout:3076 punch from 0x1b24275 to 0x1b26a66, (0x27f1 bytes) 2024-09-22T01:01:48.812 INFO:teuthology.orchestra.run.smithi002.stdout:3078 punch from 0xcb4c39 to 0xcb504c, (0x413 bytes) 2024-09-22T01:01:48.812 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:48.843 INFO:teuthology.orchestra.run.smithi002.stdout:3080 write 0x95d646d thru 0x95e06d1 (0xa265 bytes) 2024-09-22T01:01:48.850 INFO:teuthology.orchestra.run.smithi002.stdout:3081 read 0x322bd3e thru 0x323507c (0x933f bytes) 2024-09-22T01:01:48.850 INFO:teuthology.orchestra.run.smithi002.stdout:3082 read 0x5a68af0 thru 0x5a78329 (0xf83a bytes) 2024-09-22T01:01:48.850 INFO:teuthology.orchestra.run.smithi002.stdout:3083 punch from 0xc1e5b7 to 0xc2e080, (0xfac9 bytes) 2024-09-22T01:01:48.850 INFO:teuthology.orchestra.run.smithi002.stdout:3084 writesame 0x21fdbe9 thru 0x220d20a data_size 0x5(0xf622 bytes) 2024-09-22T01:01:48.854 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:48.850+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:48.877 INFO:teuthology.orchestra.run.smithi002.stdout:3085 trunc from 0x95e06d2 to 0xd2e463f 2024-09-22T01:01:48.917 INFO:teuthology.orchestra.run.smithi002.stdout:3086 trunc from 0xd2e463f to 0x2b5c6f0 2024-09-22T01:01:48.933 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:48.934 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:48.943 INFO:teuthology.orchestra.run.smithi002.stdout:3087 read 0x8f5b15 thru 0x9046a7 (0xeb93 bytes) 2024-09-22T01:01:48.949 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:48.954 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:48.951+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:48.986 INFO:teuthology.orchestra.run.smithi002.stdout:3089 write 0xeb33141 thru 0xeb35480 (0x2340 bytes) 2024-09-22T01:01:48.991 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:48.991 INFO:teuthology.orchestra.run.smithi002.stdout:3093 write 0x7735415 thru 0x774219a (0xcd86 bytes) 2024-09-22T01:01:48.996 INFO:teuthology.orchestra.run.smithi002.stdout:3094 write 0x2150252 thru 0x215704c (0x6dfb bytes) 2024-09-22T01:01:48.999 INFO:teuthology.orchestra.run.smithi002.stdout:3095 write 0x1032e90 thru 0x1036e80 (0x3ff1 bytes) 2024-09-22T01:01:49.003 INFO:teuthology.orchestra.run.smithi002.stdout:3096 read 0x7820880 thru 0x7820976 (0xf7 bytes) 2024-09-22T01:01:49.006 INFO:teuthology.orchestra.run.smithi002.stdout:3098 trunc from 0xeb35481 to 0xdb39be4 2024-09-22T01:01:49.022 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:49.022 INFO:teuthology.orchestra.run.smithi002.stdout:3100 read 0x6c4f420 thru 0x6c57a0d (0x85ee bytes) 2024-09-22T01:01:49.022 INFO:teuthology.orchestra.run.smithi002.stdout:3101 read 0xe1ebe5 thru 0xe20d2c (0x2148 bytes) 2024-09-22T01:01:49.022 INFO:teuthology.orchestra.run.smithi002.stdout:3103 read 0x7223139 thru 0x722c848 (0x9710 bytes) 2024-09-22T01:01:49.022 INFO:teuthology.orchestra.run.smithi002.stdout:3104 read 0x6eb5a7b thru 0x6ec4f2e (0xf4b4 bytes) 2024-09-22T01:01:49.022 INFO:teuthology.orchestra.run.smithi002.stdout:3105 write 0x690e8e0 thru 0x691d0ec (0xe80d bytes) 2024-09-22T01:01:49.043 INFO:teuthology.orchestra.run.smithi002.stdout:3108 trunc from 0xdb39be4 to 0x7ffa084 2024-09-22T01:01:49.056 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:49.051+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:49.072 INFO:teuthology.orchestra.run.smithi002.stdout:3109 write 0xcf122b thru 0xcfe4f8 (0xd2ce bytes) 2024-09-22T01:01:49.076 INFO:teuthology.orchestra.run.smithi002.stdout:3111 punch from 0x7e06a76 to 0x7e09c75, (0x31ff bytes) 2024-09-22T01:01:49.088 INFO:teuthology.orchestra.run.smithi002.stdout:3112 trunc from 0x7ffa084 to 0x29ffd79 2024-09-22T01:01:49.103 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:49.103 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:49.118 INFO:teuthology.orchestra.run.smithi002.stdout:3114 punch from 0x26f06d4 to 0x26f7551, (0x6e7d bytes) 2024-09-22T01:01:49.119 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:49.122 INFO:teuthology.orchestra.run.smithi002.stdout:3115 trunc from 0x29ffd79 to 0x5169364 2024-09-22T01:01:49.156 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:49.155+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:49.161 INFO:teuthology.orchestra.run.smithi002.stdout:3116 write 0xd476096 thru 0xd4760ca (0x35 bytes) 2024-09-22T01:01:49.166 INFO:teuthology.orchestra.run.smithi002.stdout:3117 write 0x33de46c thru 0x33e02aa (0x1e3f bytes) 2024-09-22T01:01:49.171 INFO:teuthology.orchestra.run.smithi002.stdout:3118 write 0x92574fb thru 0x926068e (0x9194 bytes) 2024-09-22T01:01:49.176 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:01:49.178 INFO:teuthology.orchestra.run.smithi002.stdout:3119 punch from 0x71aeccb to 0x71ba238, (0xb56d bytes) 2024-09-22T01:01:49.178 INFO:teuthology.orchestra.run.smithi002.stdout:3121 punch from 0x1d18459 to 0x1d21f4b, (0x9af2 bytes) 2024-09-22T01:01:49.178 INFO:teuthology.orchestra.run.smithi002.stdout:3122 punch from 0x1c0b8cc to 0x1c11212, (0x5946 bytes) 2024-09-22T01:01:49.178 INFO:teuthology.orchestra.run.smithi002.stdout:3123 read 0xa13a4c4 thru 0xa13e6db (0x4218 bytes) 2024-09-22T01:01:49.178 INFO:teuthology.orchestra.run.smithi002.stdout:3124 read 0xc24af23 thru 0xc24e594 (0x3672 bytes) 2024-09-22T01:01:49.178 INFO:teuthology.orchestra.run.smithi002.stdout:3125 punch from 0x51cc638 to 0x51cd8b3, (0x127b bytes) 2024-09-22T01:01:49.179 INFO:teuthology.orchestra.run.smithi002.stdout:3126 read 0xcc3b408 thru 0xcc40b0d (0x5706 bytes) 2024-09-22T01:01:49.179 INFO:teuthology.orchestra.run.smithi002.stdout:3128 writesame 0x52161 thru 0x54ce9 data_size 0x5(0x2b89 bytes) 2024-09-22T01:01:49.180 INFO:teuthology.orchestra.run.smithi002.stdout:3129 read 0x9e252c5 thru 0x9e2b255 (0x5f91 bytes) 2024-09-22T01:01:49.182 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:49.202 INFO:teuthology.orchestra.run.smithi002.stdout:3130 trunc from 0xd4760cb to 0x116266e 2024-09-22T01:01:49.242 INFO:teuthology.orchestra.run.smithi002.stdout:3132 write 0x17cc10b thru 0x17d4c78 (0x8b6e bytes) 2024-09-22T01:01:49.246 INFO:teuthology.orchestra.run.smithi002.stdout:3133 read 0xa75954 thru 0xa7f54e (0x9bfb bytes) 2024-09-22T01:01:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:3134 read 0x1cced5 thru 0x1dcd3c (0xfe68 bytes) 2024-09-22T01:01:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:3137 read 0x81a1ac thru 0x825a68 (0xb8bd bytes) 2024-09-22T01:01:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:3138 read 0xa84660 thru 0xa8bef7 (0x7898 bytes) 2024-09-22T01:01:49.249 INFO:teuthology.orchestra.run.smithi002.stdout:3140 write 0x19e616 thru 0x1a1c3f (0x362a bytes) 2024-09-22T01:01:49.257 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:49.254+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:49.265 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:49.265 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:49.273 INFO:teuthology.orchestra.run.smithi002.stdout:3141 write 0x6546d5f thru 0x654c56e (0x5810 bytes) 2024-09-22T01:01:49.283 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:49.299 INFO:teuthology.orchestra.run.smithi002.stdout:3143 writesame 0xa347b21 thru 0xa347e3d data_size 0x1(0x31d bytes) 2024-09-22T01:01:49.299 INFO:teuthology.orchestra.run.smithi002.stdout:3144 flatten 2024-09-22T01:01:49.326 INFO:teuthology.orchestra.run.smithi002.stdout:3145 write 0xadcf964 thru 0xaddd50a (0xdba7 bytes) 2024-09-22T01:01:49.332 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:49.335 INFO:teuthology.orchestra.run.smithi002.stdout:3148 trunc from 0xaddd50b to 0x8bb1a18 2024-09-22T01:01:49.342 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:49.356 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:49.356 INFO:teuthology.orchestra.run.smithi002.stdout:3150 read 0x84e5d7 thru 0x852932 (0x435c bytes) 2024-09-22T01:01:49.358 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:49.355+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:49.373 INFO:teuthology.orchestra.run.smithi002.stdout:3152 write 0xba0df92 thru 0xba15c5f (0x7cce bytes) 2024-09-22T01:01:49.374 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:49.374 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:49.388 INFO:teuthology.orchestra.run.smithi002.stdout:3153 trunc from 0xba15c60 to 0x531c1bb 2024-09-22T01:01:49.392 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:49.392 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:49.413 INFO:teuthology.orchestra.run.smithi002.stdout:3155 trunc from 0x531c1bb to 0x3ad75d9 2024-09-22T01:01:49.424 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:49.424 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:49.426 INFO:teuthology.orchestra.run.smithi002.stdout:3157 read 0x144a46c thru 0x1459383 (0xef18 bytes) 2024-09-22T01:01:49.427 INFO:teuthology.orchestra.run.smithi002.stdout:3158 read 0x3604e58 thru 0x3613901 (0xeaaa bytes) 2024-09-22T01:01:49.429 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:49.429 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:49.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:38846 3.1 3:823bb6bd:::rbd_data.117c55693b3.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.941523074999999, 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049688678999999999, 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 38846 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487015+0000", 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.487020+0000", 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.023e-06 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497105+0000", 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010085393 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497147+0000", 2024-09-22T01:01:49.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1866999999999997e-05 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497210+0000", 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2324999999999997e-05 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497457+0000", 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024715099999999998 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.497726+0000", 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026944000000000002 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.498317+0000", 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059106199999999995 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.516775+0000", 2024-09-22T01:01:49.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018457952 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536527+0000", 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019751484 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536622+0000", 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5529e-05 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:26.536704+0000", 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1452999999999994e-05 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55415 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 503808~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.497733+0000", 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9308054979999998, 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048933005000000002, 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55415 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497733+0000", 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497733+0000", 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497733+0000", 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1199999999999999e-07 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497734+0000", 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.018e-06 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497737+0000", 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.022e-06 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497739+0000", 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4940000000000002e-06 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544791+0000", 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047051608000000002 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5760999999999993e-05 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546666+0000", 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00177889 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.440 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55527 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 18432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.497742+0000", 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.930795636, 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047598541000000001, 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55527 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497742+0000", 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497742+0000", 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497743+0000", 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1299999999999999e-07 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497744+0000", 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0610000000000001e-06 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497747+0000", 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497750+0000", 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5160000000000001e-06 2024-09-22T01:01:49.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545103+0000", 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047353108999999997 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545196+0000", 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3323000000000001e-05 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545341+0000", 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145029 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55528 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.497753+0000", 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9307851170000001, 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047967865999999998, 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55528 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497753+0000", 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.497753+0000", 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498606+0000", 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00085334300000000001 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498610+0000", 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7629999999999998e-06 2024-09-22T01:01:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498623+0000", 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2632999999999999e-05 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498631+0000", 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0180000000000001e-06 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545371+0000", 2024-09-22T01:01:49.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046739979000000001 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545459+0000", 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8158000000000005e-05 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545721+0000", 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026197200000000002 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55529 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 75776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498663+0000", 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9298749940000004, 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047357508999999999, 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55529 2024-09-22T01:01:49.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498663+0000", 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498663+0000", 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498664+0000", 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498675+0000", 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1565e-05 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498685+0000", 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0139999999999999e-05 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498692+0000", 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545767+0000", 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047074963999999997 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545880+0000", 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112509 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546021+0000", 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000141116 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55530 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498725+0000", 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9298129980000001, 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047552522, 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55530 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498725+0000", 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498725+0000", 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498726+0000", 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498729+0000", 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.162e-06 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498737+0000", 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1589999999999993e-06 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498743+0000", 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8610000000000003e-06 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546047+0000", 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047303975999999998 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546134+0000", 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7113000000000004e-05 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546278+0000", 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014368200000000001 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55531 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 133120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498752+0000", 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9297863419999999, 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047885688000000003, 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55531 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498752+0000", 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498752+0000", 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498752+0000", 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3000000000000001e-07 2024-09-22T01:01:49.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498755+0000", 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3609999999999999e-06 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498762+0000", 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8730000000000001e-06 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498767+0000", 2024-09-22T01:01:49.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4269999999999999e-06 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047542327000000002 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546397+0000", 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7410999999999994e-05 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546637+0000", 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000240759 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55532 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498777+0000", 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.929760667, 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048067331999999997, 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55532 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498777+0000", 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498777+0000", 2024-09-22T01:01:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498778+0000", 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498781+0000", 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.599e-06 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498788+0000", 2024-09-22T01:01:49.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3590000000000002e-06 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498793+0000", 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4260000000000002e-06 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546716+0000", 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047922431000000001 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546781+0000", 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5140000000000003e-05 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546845+0000", 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3895999999999999e-05 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55533 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498813+0000", 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.929725092, 2024-09-22T01:01:49.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048206710999999999, 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55533 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498813+0000", 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498813+0000", 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498814+0000", 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498837+0000", 2024-09-22T01:01:49.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3594000000000001e-05 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498847+0000", 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8560000000000002e-06 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498852+0000", 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546870+0000", 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048017277999999997 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546921+0000", 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1845000000000003e-05 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547020+0000", 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8290999999999998e-05 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55534 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498882+0000", 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.929656424, 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048300441999999999, 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55534 2024-09-22T01:01:49.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498882+0000", 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498882+0000", 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498882+0000", 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7100000000000001e-07 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498885+0000", 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8009999999999999e-06 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498893+0000", 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7989999999999994e-06 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498898+0000", 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2680000000000002e-06 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547046+0000", 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048147476000000002 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547095+0000", 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9611000000000003e-05 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547182+0000", 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6716000000000001e-05 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55535 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498906+0000", 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9296322159999999, 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048432527000000003, 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55535 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498906+0000", 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498906+0000", 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498906+0000", 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498909+0000", 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6079999999999998e-06 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498915+0000", 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498920+0000", 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7249999999999997e-06 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547208+0000", 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048287494 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547259+0000", 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1632999999999998e-05 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547338+0000", 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9235999999999998e-05 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.480 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55536 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 276480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498927+0000", 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.929611403, 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049496845999999997, 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55536 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498927+0000", 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498927+0000", 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498927+0000", 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2200000000000004e-07 2024-09-22T01:01:49.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498929+0000", 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8139999999999999e-06 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498935+0000", 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8019999999999997e-06 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498939+0000", 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2320000000000003e-06 2024-09-22T01:01:49.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547364+0000", 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048425134000000002 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.547426+0000", 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1908000000000004e-05 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548424+0000", 2024-09-22T01:01:49.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099743399999999995 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55537 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.498981+0000", 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9295568239999996, 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049635472999999999, 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55537 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498981+0000", 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498981+0000", 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498982+0000", 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6499999999999999e-07 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498985+0000", 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.802e-06 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498993+0000", 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4209999999999995e-06 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.498999+0000", 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548457+0000", 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049458624 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548536+0000", 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8374999999999994e-05 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548617+0000", 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1023e-05 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55538 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 333824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.489 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499017+0000", 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9295208800000001, 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049965862999999999, 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55538 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499017+0000", 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499017+0000", 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499018+0000", 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499019+0000", 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8190000000000001e-06 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499024+0000", 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4220000000000002e-06 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499027+0000", 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1690000000000001e-06 2024-09-22T01:01:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548700+0000", 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049673292000000001 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548781+0000", 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0359999999999996e-05 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548983+0000", 2024-09-22T01:01:49.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020243799999999999 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9294937599999997, 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:01:49.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:01:49.498 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9294659169999999, 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:01:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9294337009999998, 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:01:49.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:01:49.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9294119629999997, 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:01:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9293923079999997, 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:01:49.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9293612939999996, 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:01:49.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:01:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:01:49.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:01:49.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.520 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.520 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.520 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:49.521 INFO:teuthology.orchestra.run.smithi002.stdout:3159 trunc from 0x3ad75d9 to 0x71e2a90 2024-09-22T01:01:49.521 INFO:teuthology.orchestra.run.smithi002.stdout:3160 writesame 0x3351823 thru 0x335a059 data_size 0x1(0x8837 bytes) 2024-09-22T01:01:49.522 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:49.522 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:49.522 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:49.523 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:49.523 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:49.523 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:49.524 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:49.524 INFO:teuthology.orchestra.run.smithi002.stdout:3162 punch from 0x5017534 to 0x5017f3e, (0xa0a bytes) 2024-09-22T01:01:49.524 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:49.524 INFO:teuthology.orchestra.run.smithi002.stdout:3166 trunc from 0x71e2a90 to 0xb537fa7 2024-09-22T01:01:49.524 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:49.525 INFO:teuthology.orchestra.run.smithi002.stdout:3167 clone 28 order 20 su 4096 sc 7 2024-09-22T01:01:49.525 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:49.519+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:49.555 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:01:49.555 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:01:49.555 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-22T01:01:49.555 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:01:49.555 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:01:49.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-22T01:01:49.622 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:49.619+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:49.688 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:49.688 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-22T01:01:49.688 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:49.701 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:49.701 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:49.701 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:49.701 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:49.715 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:49.744 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:49.743+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:49.824 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:49.823+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:49.835 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:49.835 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:49.835 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:49.835 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:49.835 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:49.849 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:49.925 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:49.923+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.892051756999997, 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.892032897999997, 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.892014713000002, 2024-09-22T01:01:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891996925000001, 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891978367, 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891960314999999, 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:49.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891942966000002, 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:50.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891921013000001, 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:50.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891903478000003, 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:50.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:50.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:50.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:50.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891879692000003, 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:50.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:50.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:50.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891861830000003, 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:50.029 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:50.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:50.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.034 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.034 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891834592999999, 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.035 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:50.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891815887999996, 2024-09-22T01:01:50.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:50.040 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:50.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:50.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:50.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:50.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:50.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:50.046 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891792391000003, 2024-09-22T01:01:50.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:50.046 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:50.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891768421999998, 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:50.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.891749206999997, 2024-09-22T01:01:50.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:50.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.890744656999999, 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:50.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.888291469000002, 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:50.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:50.069 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.888269389000001, 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:50.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:50.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:50.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 54.888238704000003, 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:50.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:50.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.079 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.079 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:50.080 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:50.080 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:50.074+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:50.167 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:01:50.181 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:50.174+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:50.262 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:50.262 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:50.262 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:50.262 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:50.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:50.282 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:50.279+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:50.383 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:50.378+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:50.400 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:50.400 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:50.400 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:50.400 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:50.400 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:50.415 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:50.468 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-22T01:01:50.478 INFO:teuthology.orchestra.run.smithi002.stdout:3168 writesame 0xae618bb thru 0xae6bd5b data_size 0x1(0xa4a1 bytes) 2024-09-22T01:01:50.484 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:50.483+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:50.512 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:50.512 INFO:teuthology.orchestra.run.smithi002.stdout:3170 read 0x43d7a66 thru 0x43dd45a (0x59f5 bytes) 2024-09-22T01:01:50.512 INFO:teuthology.orchestra.run.smithi002.stdout:3172 read 0x50be7ea thru 0x50c6681 (0x7e98 bytes) 2024-09-22T01:01:50.512 INFO:teuthology.orchestra.run.smithi002.stdout:3173 write 0x148db08 thru 0x148f655 (0x1b4e bytes) 2024-09-22T01:01:50.585 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:50.578+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:50.604 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:50.604 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8964 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'20, mlcod=32'17)", 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.464607608999998, 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077894110000000004, 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:50.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136525+0000", 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3540000000000001e-06 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137650+0000", 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011257190000000001 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137671+0000", 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0560999999999998e-05 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137680+0000", 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137763+0000", 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2472999999999995e-05 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137793+0000", 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-05 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144267+0000", 2024-09-22T01:01:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064739139999999999 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144313+0000", 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5624999999999998e-05 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8968 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'21, mlcod=32'17)", 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.460181564000003, 2024-09-22T01:01:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004894771, 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140952+0000", 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1849999999999998e-06 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.141985+0000", 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001032984 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142001+0000", 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6776999999999999e-05 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142010+0000", 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4640000000000006e-06 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142101+0000", 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1312999999999998e-05 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142119+0000", 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8202e-05 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145810+0000", 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036909970000000001 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145844+0000", 2024-09-22T01:01:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3849000000000001e-05 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9077 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'31, mlcod=32'29)", 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.205857189, 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034796760000000001, 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395277+0000", 2024-09-22T01:01:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8600000000000001e-06 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395294+0000", 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7513e-05 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395337+0000", 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2749e-05 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395358+0000", 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1097000000000001e-05 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395548+0000", 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189829 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395593+0000", 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4919000000000001e-05 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398634+0000", 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003041373 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398753+0000", 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119336 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35894 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'145, mlcod=47'143)", 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.448116312, 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055540160000000002, 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8700000000000002e-07 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153852+0000", 2024-09-22T01:01:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083735000000000005 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153865+0000", 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2804e-05 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153872+0000", 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153943+0000", 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0908000000000006e-05 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153968+0000", 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5530000000000001e-05 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158535+0000", 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045670559999999999 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158569+0000", 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210999999999998e-05 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35895 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'147, mlcod=47'143)", 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.445316156000001, 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031074570000000001, 2024-09-22T01:01:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155815+0000", 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155816+0000", 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.666e-06 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155839+0000", 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2768e-05 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155940+0000", 2024-09-22T01:01:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010056999999999999 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.155982+0000", 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1850000000000001e-05 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.156671+0000", 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00068920600000000004 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.156712+0000", 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0917999999999997e-05 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158885+0000", 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021735859999999999 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158922+0000", 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6893000000000001e-05 2024-09-22T01:01:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37899 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'73, mlcod=48'71)", 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.421612489000001, 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012095786000000001, 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179519+0000", 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4099999999999997e-07 2024-09-22T01:01:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183241+0000", 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037216110000000001 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183261+0000", 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9799000000000001e-05 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183274+0000", 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3046e-05 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183374+0000", 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010009399999999999 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183399+0000", 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5375999999999999e-05 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191540+0000", 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081403029999999998 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191614+0000", 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4715999999999994e-05 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37936 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'74, mlcod=48'72)", 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.251851418000001, 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003111749, 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349280+0000", 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349282+0000", 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2970000000000002e-06 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349299+0000", 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7278999999999998e-05 2024-09-22T01:01:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349331+0000", 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2218000000000002e-05 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349346+0000", 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4856e-05 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349548+0000", 2024-09-22T01:01:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020148599999999999 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.349608+0000", 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9893999999999999e-05 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.352260+0000", 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002652583 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.352391+0000", 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013113600000000001 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:43527 3.9 e60/22 3:96fe8936:::rbd_data.117c83c8bd76.000000000000001b:head v 60'182, mlcod=60'180)", 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.759561278, 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005051725, 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841572+0000", 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845187+0000", 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036148880000000001 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845210+0000", 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2926e-05 2024-09-22T01:01:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845221+0000", 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.17e-05 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845322+0000", 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101003 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845348+0000", 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5361e-05 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846545+0000", 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011971659999999999 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846621+0000", 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6631000000000005e-05 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:53802 3.13 e66/65 3:cb3ff6d1:::rbd_data.117c93605fe7.00000000000000ed:head v 66'110, mlcod=0'0)", 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1638704430000004, 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003122391, 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437260+0000", 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437261+0000", 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4799999999999998e-07 2024-09-22T01:01:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437266+0000", 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437276+0000", 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6420000000000002e-06 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.437283+0000", 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9450000000000004e-06 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.438702+0000", 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014196409999999999 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.438725+0000", 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2456e-05 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.440350+0000", 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001624955 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.440383+0000", 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3204999999999998e-05 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56176 3.9 3:91eeadac:::rbd_data.117c93605fe7.00000000000001f7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.702376975, 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038039409999999999, 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56176 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898756+0000", 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.832e-06 2024-09-22T01:01:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898763+0000", 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3440000000000003e-06 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898778+0000", 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4982e-05 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898787+0000", 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7479999999999999e-06 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898845+0000", 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8616999999999998e-05 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898903+0000", 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7623e-05 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898982+0000", 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9124999999999999e-05 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.899559+0000", 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00057709899999999997 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902402+0000", 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028426990000000002 2024-09-22T01:01:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902461+0000", 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8607999999999999e-05 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902558+0000", 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7263999999999996e-05 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56184 3.9 3:914e0854:::rbd_data.117c93605fe7.000000000000021b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.6997685320000002, 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040309569999999999, 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56184 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901364+0000", 2024-09-22T01:01:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.48e-06 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901369+0000", 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1050000000000003e-06 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901384+0000", 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5098e-05 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901392+0000", 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9969999999999998e-06 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901444+0000", 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2268999999999997e-05 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901496+0000", 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1864999999999999e-05 2024-09-22T01:01:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901562+0000", 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5718999999999999e-05 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902103+0000", 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054061300000000001 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905243+0000", 2024-09-22T01:01:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031400059999999999 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905305+0000", 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2527000000000006e-05 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905393+0000", 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8277999999999997e-05 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56364 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [write 36377~4583] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.5564724689999991, 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034458259999999999, 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56364 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044660+0000", 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.046e-06 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044672+0000", 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2119999999999999e-05 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044691+0000", 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9320000000000001e-05 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044698+0000", 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2420000000000003e-06 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044756+0000", 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7547999999999999e-05 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044836+0000", 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0657999999999999e-05 2024-09-22T01:01:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044966+0000", 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000129682 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.046764+0000", 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001797877 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.047933+0000", 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001169139 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.047999+0000", 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5537000000000006e-05 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.048104+0000", 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010565699999999999 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56402 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.5259202149999993, 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004133734, 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56402 2024-09-22T01:01:50.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075212+0000", 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7060000000000001e-06 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075220+0000", 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2590000000000003e-06 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075235+0000", 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.537e-05 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075243+0000", 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1159999999999999e-06 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075314+0000", 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0535000000000001e-05 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075399+0000", 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5745e-05 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075512+0000", 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112575 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.076225+0000", 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071304799999999998 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079253+0000", 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030281520000000001 2024-09-22T01:01:50.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079290+0000", 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7240000000000003e-05 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079344+0000", 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3987999999999999e-05 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:58441 3.9 3:961db89f:::rbd_data.117c93605fe7.000000000000027c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8592600810000004, 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035419259999999999, 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 58441 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.671 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741873+0000", 2024-09-22T01:01:50.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7990000000000001e-06 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741879+0000", 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0249999999999999e-06 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741896+0000", 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6962000000000001e-05 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741904+0000", 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4489999999999999e-06 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741961+0000", 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7206000000000001e-05 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742048+0000", 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7078999999999998e-05 2024-09-22T01:01:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742156+0000", 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010733799999999999 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742877+0000", 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072088000000000002 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745306+0000", 2024-09-22T01:01:50.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002429512 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745365+0000", 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8962000000000002e-05 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745413+0000", 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7713999999999998e-05 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59024 3.9 3:935ab33d:::rbd_data.117ce35b6780.00000000000001db:head [set-alloc-hint object_size 262144 write_size 262144,write 33792~3072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.883545281, 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048336580000000002, 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59024 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717588+0000", 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8030000000000001e-06 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717600+0000", 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2028000000000001e-05 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717626+0000", 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5930000000000001e-05 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717639+0000", 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2296000000000001e-05 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717728+0000", 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9091999999999993e-05 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717864+0000", 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013604200000000001 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.718008+0000", 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014423800000000001 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.719029+0000", 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001020897 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722322+0000", 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032925509999999999 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722370+0000", 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8755000000000003e-05 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722419+0000", 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9026000000000001e-05 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.682 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59026 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [set-alloc-hint object_size 262144 write_size 262144,write 32768~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8808761860000001, 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046681709999999996, 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59026 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720256+0000", 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3969999999999999e-06 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720267+0000", 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699e-05 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720285+0000", 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8561999999999999e-05 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720296+0000", 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0324e-05 2024-09-22T01:01:50.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720362+0000", 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6156999999999997e-05 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720474+0000", 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111656 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720641+0000", 2024-09-22T01:01:50.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167875 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.721449+0000", 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00080787999999999997 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724779+0000", 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033294090000000002 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724864+0000", 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5535e-05 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724923+0000", 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8677000000000001e-05 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59035 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:50.688 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8619382160000004, 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056261260000000004, 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59035 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739194+0000", 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7599999999999996e-07 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739199+0000", 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1919999999999996e-06 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739212+0000", 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3484e-05 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739222+0000", 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3279999999999995e-06 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739864+0000", 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064251300000000005 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739973+0000", 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010924700000000001 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740123+0000", 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014958199999999999 2024-09-22T01:01:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740800+0000", 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067721600000000002 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744735+0000", 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039345889999999996 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744779+0000", 2024-09-22T01:01:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4196999999999997e-05 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744819+0000", 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0002000000000001e-05 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:59069 3.1a e70/67)", 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7882470440000002, 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033876940000000001, 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812885+0000", 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-06 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812930+0000", 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4731999999999998e-05 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812963+0000", 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3791999999999998e-05 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812986+0000", 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2493e-05 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.814703+0000", 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017169799999999999 2024-09-22T01:01:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.814754+0000", 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1035999999999997e-05 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.816213+0000", 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014592100000000001 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.816272+0000", 2024-09-22T01:01:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8446999999999998e-05 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59186 3.9 3:973f8b5e:::rbd_data.117ce35b6780.000000000000007d:head [set-alloc-hint object_size 262144 write_size 262144,write 131217~7023] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6030751839999997, 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039873030000000002, 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59186 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998059+0000", 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998076+0000", 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.732e-05 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998110+0000", 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3785999999999998e-05 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998124+0000", 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4544e-05 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998226+0000", 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101544 2024-09-22T01:01:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998436+0000", 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021000899999999999 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998697+0000", 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000261406 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.999737+0000", 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010393519999999999 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001927+0000", 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021903420000000001 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001985+0000", 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7711999999999999e-05 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.002043+0000", 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8477999999999998e-05 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59243 3.9 3:973f8b5e:::rbd_data.117ce35b6780.000000000000007d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5243926949999995, 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032113020000000001, 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59243 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076741+0000", 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0769999999999999e-06 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076751+0000", 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3929999999999997e-06 2024-09-22T01:01:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076778+0000", 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7001000000000001e-05 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076790+0000", 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.183e-05 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076875+0000", 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5158e-05 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076967+0000", 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2660999999999999e-05 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.077078+0000", 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110343 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.078598+0000", 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00152005 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079834+0000", 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001236343 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079897+0000", 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2724000000000005e-05 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079950+0000", 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2722000000000001e-05 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:50.711 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:50.713 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:50.713 INFO:teuthology.orchestra.run.smithi002.stdout:3174 read 0x249b9c6 thru 0x24a086c (0x4ea7 bytes) 2024-09-22T01:01:50.713 INFO:teuthology.orchestra.run.smithi002.stdout:3175 write 0x6e55776 thru 0x6e56ce3 (0x156e bytes) 2024-09-22T01:01:50.713 INFO:teuthology.orchestra.run.smithi002.stdout:3176 read 0xa00f486 thru 0xa01a78e (0xb309 bytes) 2024-09-22T01:01:50.713 INFO:teuthology.orchestra.run.smithi002.stdout:3177 read 0x68a466e thru 0x68af149 (0xaadc bytes) 2024-09-22T01:01:50.714 INFO:teuthology.orchestra.run.smithi002.stdout:3178 trunc from 0xb537fa7 to 0x68b2ab6 2024-09-22T01:01:50.714 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:50.711+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:50.719 INFO:teuthology.orchestra.run.smithi002.stdout:3181 writesame 0x67d68df thru 0x67e0545 data_size 0x1(0x9c67 bytes) 2024-09-22T01:01:50.737 INFO:teuthology.orchestra.run.smithi002.stdout:3183 read 0x4c028b3 thru 0x4c033c5 (0xb13 bytes) 2024-09-22T01:01:50.737 INFO:teuthology.orchestra.run.smithi002.stdout:3185 write 0x4676897 thru 0x467d111 (0x687b bytes) 2024-09-22T01:01:50.814 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:50.811+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:50.851 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:50.851 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:50.851 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:50.852 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:50.866 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:50.958 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:50.954+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:50.962 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:50.962 INFO:teuthology.orchestra.run.smithi002.stdout:3188 read 0x43f27a7 thru 0x43fa26d (0x7ac7 bytes) 2024-09-22T01:01:50.962 INFO:teuthology.orchestra.run.smithi002.stdout:3191 read 0x5b4d1fb thru 0x5b5a094 (0xce9a bytes) 2024-09-22T01:01:50.962 INFO:teuthology.orchestra.run.smithi002.stdout:3195 punch from 0x648a7a2 to 0x64903d2, (0x5c30 bytes) 2024-09-22T01:01:50.962 INFO:teuthology.orchestra.run.smithi002.stdout:3196 write 0x67417e6 thru 0x6742e72 (0x168d bytes) 2024-09-22T01:01:50.980 INFO:teuthology.orchestra.run.smithi002.stdout:3197 write 0x8ba25ef thru 0x8bb03a9 (0xddbb bytes) 2024-09-22T01:01:51.003 INFO:teuthology.orchestra.run.smithi002.stdout:3198 write 0x95070f8 thru 0x950c365 (0x526e bytes) 2024-09-22T01:01:51.015 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:51.011+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:51.016 INFO:teuthology.orchestra.run.smithi002.stdout:3199 read 0x4a04743 thru 0x4a0b544 (0x6e02 bytes) 2024-09-22T01:01:51.024 INFO:teuthology.orchestra.run.smithi002.stdout:3200 trunc from 0x950c366 to 0xea8da2a 2024-09-22T01:01:51.035 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:51.035 INFO:teuthology.orchestra.run.smithi002.stdout:3205 read 0xcab3ef5 thru 0xcab3fe1 (0xed bytes) 2024-09-22T01:01:51.035 INFO:teuthology.orchestra.run.smithi002.stdout:3206 read 0x75215c3 thru 0x752e33e (0xcd7c bytes) 2024-09-22T01:01:51.036 INFO:teuthology.orchestra.run.smithi002.stdout:3207 writesame 0x41693c2 thru 0x4176764 data_size 0x1(0xd3a3 bytes) 2024-09-22T01:01:51.057 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:51.057 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:51.057 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:51.057 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:51.057 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:51.073 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:51.118 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:51.114+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:51.144 INFO:teuthology.orchestra.run.smithi002.stdout:3208 read 0x2f47432 thru 0x2f55598 (0xe167 bytes) 2024-09-22T01:01:51.150 INFO:teuthology.orchestra.run.smithi002.stdout:3209 writesame 0x934831 thru 0x93f0e1 data_size 0x5(0xa8b1 bytes) 2024-09-22T01:01:51.217 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:51.215+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:51.280 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:51.280 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:51.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:51.280 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:51.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.280 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.280 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:51.280 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8053739719999999, 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:01:51.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8053531429999996, 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:01:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8053246279999993, 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.290 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:01:51.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8053019779999993, 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:01:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8052799830000001, 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:01:51.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8052549189999993, 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.307 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8052324859999995, 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:01:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8052110090000006, 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:01:51.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8051822509999997, 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:01:51.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8043692589999996, 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:01:51.321 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:01:51.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.325 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8043445739999999, 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:01:51.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8000230399999992, 2024-09-22T01:01:51.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:01:51.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:01:51.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7999941889999995, 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:01:51.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:51.339 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7999724639999997, 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:01:51.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7999520340000004, 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:01:51.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7999297179999996, 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:01:51.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7998619229999999, 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:01:51.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7998287859999991, 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.357 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:51.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.361 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7998048640000004, 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:01:51.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7997812670000002, 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:51.366 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:01:51.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:01:51.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:51.370 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:51.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:51.372 INFO:teuthology.orchestra.run.smithi002.stdout:3210 read 0x2f15ac0 thru 0x2f1da5a (0x7f9b bytes) 2024-09-22T01:01:51.372 INFO:teuthology.orchestra.run.smithi002.stdout:3213 punch from 0xa05212e to 0xa0588d9, (0x67ab bytes) 2024-09-22T01:01:51.372 INFO:teuthology.orchestra.run.smithi002.stdout:3214 clone 29 order 23 su 1048576 sc 2 2024-09-22T01:01:51.373 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:51.371+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:51.472 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:51.471+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:51.499 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:51.500 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:51.500 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:51.500 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:51.514 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:51.530 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:51.600 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:51.599+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:51.674 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:51.670+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:51.680 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:51.680 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:51.680 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:51.680 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:51.680 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:51.686 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:51.686 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:51.694 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:51.703 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:51.721 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:51.738 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:51.738 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:51.756 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:51.774 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:51.774 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:51.784 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:51.783+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:51.792 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:51.811 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:51.862 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:51.862 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:51.875 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:51.870+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:51.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:51.976 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:51.971+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:52.045 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:52.045 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:52.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:52.076 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:52.070+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:52.177 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:52.174+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:52.194 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:01:52.224 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:52.224 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:52.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:52.284 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:52.283+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:52.304 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone28 from 0xea8da2a (overlap 0x68b2ab6) to 0x0 2024-09-22T01:01:52.378 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:52.375+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:52.380 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:52.380 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:52.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:52.484 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:52.483+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:52.581 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:52.579+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:52.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:52.720 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:52.719+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:52.783 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:52.779+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:52.833 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:52.833 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:52.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:52.885 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:52.882+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:52.984 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:52.983+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:53.029 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:53.043 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:53.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:53.086 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-22T01:01:53.092 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:53.091+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:53.187 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:53.182+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:53.199 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:53.199 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:01:53.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:01:53.288 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:53.282+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:53.332 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:53.332 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:53.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:01:53.388 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:53.387+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:53.463 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:53.463 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:53.476 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:53.485 INFO:teuthology.orchestra.run.smithi002.stdout:3215 trunc from 0xea8da2a to 0x3d537fc 2024-09-22T01:01:53.489 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:53.482+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:53.586 INFO:teuthology.orchestra.run.smithi002.stdout:3216 write 0x8401033 thru 0x840ef8b (0xdf59 bytes) 2024-09-22T01:01:53.590 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:53.586+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:53.593 INFO:teuthology.orchestra.run.smithi002.stdout:3217 write 0x3de8707 thru 0x3df07f1 (0x80eb bytes) 2024-09-22T01:01:53.610 INFO:teuthology.orchestra.run.smithi002.stdout:3218 write 0xbb03a46 thru 0xbb0600f (0x25ca bytes) 2024-09-22T01:01:53.615 INFO:teuthology.orchestra.run.smithi002.stdout:3220 clone 30 order 22 su 1048576 sc 12 2024-09-22T01:01:53.621 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:53.621 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:53.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:53.692 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:53.687+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:53.792 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:53.791+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:53.797 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:53.797 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:53.811 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:53.828 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:53.893 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:53.891+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:53.940 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:53.940 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:53.956 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:53.999 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:53.999 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:53.999 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:53.994+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:54.016 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:54.034 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:54.052 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:54.052 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:54.074 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:54.074 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:54.079 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:54.079 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:54.093 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:54.095 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:54.095 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:54.096 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:54.090+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:54.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:54.117 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:54.139 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:54.139 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:54.201 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:54.198+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:54.214 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:54.214 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:54.214 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:54.214 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:54.214 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:54.229 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:54.318 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:54.314+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:54.323 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone29 from 0xbb06010 (overlap 0x3d537fc) to 0x2aeda63 2024-09-22T01:01:54.397 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:54.395+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:54.403 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:54.403 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:54.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:54.403 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:54.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.403 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.403 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322463929000001, 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:54.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:54.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322445070000001, 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.408 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.412 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322426884999999, 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322409096999998, 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:54.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:54.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322390538999997, 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.421 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322372487000003, 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.429 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322355137999999, 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:54.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322333184999998, 2024-09-22T01:01:54.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:54.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.438 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.32231565, 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:54.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322291864, 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:54.443 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:54.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322274002, 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.448 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:54.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322246765000003, 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:54.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:54.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.32222806, 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.457 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:54.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322204563, 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:54.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.465 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322180594000002, 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:54.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.322161379000001, 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.321156829000003, 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.475 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:54.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.318703640999999, 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.318681560999998, 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.484 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 59.318650876, 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:54.488 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:54.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.492 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:54.493 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:54.538 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:54.534+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:54.598 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:54.595+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:54.665 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:54.665 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:54.665 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:54.665 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:54.679 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:54.702 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:01:54.703 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:01:54.703 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-22T01:01:54.703 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:01:54.730 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:54.726+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:54.800 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:54.794+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:54.807 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:54.807 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:54.807 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:54.807 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:54.807 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:54.821 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:54.847 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-22T01:01:54.850 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:54.850 INFO:teuthology.orchestra.run.smithi002.stdout:3222 writesame 0x163c971 thru 0x1647b18 data_size 0x5(0xb1a8 bytes) 2024-09-22T01:01:54.901 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:54.899+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:54.962 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8964 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'20, mlcod=32'17)", 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.824306841000002, 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077894110000000004, 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:54.964 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:54.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:54.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136525+0000", 2024-09-22T01:01:54.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3540000000000001e-06 2024-09-22T01:01:54.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137650+0000", 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011257190000000001 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137671+0000", 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0560999999999998e-05 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137680+0000", 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137763+0000", 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2472999999999995e-05 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137793+0000", 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-05 2024-09-22T01:01:54.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144267+0000", 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064739139999999999 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144313+0000", 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5624999999999998e-05 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.969 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8968 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'21, mlcod=32'17)", 2024-09-22T01:01:54.969 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:54.969 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.819880796, 2024-09-22T01:01:54.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004894771, 2024-09-22T01:01:54.969 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:54.970 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:54.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140952+0000", 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1849999999999998e-06 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.141985+0000", 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001032984 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142001+0000", 2024-09-22T01:01:54.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6776999999999999e-05 2024-09-22T01:01:54.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142010+0000", 2024-09-22T01:01:54.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4640000000000006e-06 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142101+0000", 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1312999999999998e-05 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142119+0000", 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8202e-05 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145810+0000", 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036909970000000001 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145844+0000", 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3849000000000001e-05 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9077 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'31, mlcod=32'29)", 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.565556420999997, 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034796760000000001, 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:54.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395277+0000", 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8600000000000001e-06 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395294+0000", 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7513e-05 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395337+0000", 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2749e-05 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395358+0000", 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1097000000000001e-05 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395548+0000", 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189829 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395593+0000", 2024-09-22T01:01:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4919000000000001e-05 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398634+0000", 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003041373 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398753+0000", 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119336 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35894 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'145, mlcod=47'143)", 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.807815544, 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055540160000000002, 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8700000000000002e-07 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153852+0000", 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083735000000000005 2024-09-22T01:01:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153865+0000", 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2804e-05 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153872+0000", 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153943+0000", 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0908000000000006e-05 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153968+0000", 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5530000000000001e-05 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158535+0000", 2024-09-22T01:01:54.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045670559999999999 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158569+0000", 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210999999999998e-05 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37899 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'73, mlcod=48'71)", 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.781311721000002, 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012095786000000001, 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179519+0000", 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4099999999999997e-07 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183241+0000", 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037216110000000001 2024-09-22T01:01:54.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183261+0000", 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9799000000000001e-05 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183274+0000", 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3046e-05 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183374+0000", 2024-09-22T01:01:54.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010009399999999999 2024-09-22T01:01:54.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183399+0000", 2024-09-22T01:01:54.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5375999999999999e-05 2024-09-22T01:01:54.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:54.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191540+0000", 2024-09-22T01:01:54.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081403029999999998 2024-09-22T01:01:54.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191614+0000", 2024-09-22T01:01:54.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4715999999999994e-05 2024-09-22T01:01:54.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:43527 3.9 e60/22 3:96fe8936:::rbd_data.117c83c8bd76.000000000000001b:head v 60'182, mlcod=60'180)", 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.11926051, 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005051725, 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:54.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:54.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841572+0000", 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845187+0000", 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036148880000000001 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845210+0000", 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2926e-05 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845221+0000", 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.17e-05 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845322+0000", 2024-09-22T01:01:54.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101003 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845348+0000", 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5361e-05 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846545+0000", 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011971659999999999 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846621+0000", 2024-09-22T01:01:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6631000000000005e-05 2024-09-22T01:01:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56176 3.9 3:91eeadac:::rbd_data.117c93605fe7.00000000000001f7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.062076207, 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038039409999999999, 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56176 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:54.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:54.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:54.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898756+0000", 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.832e-06 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898763+0000", 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3440000000000003e-06 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898778+0000", 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4982e-05 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898787+0000", 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7479999999999999e-06 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898845+0000", 2024-09-22T01:01:54.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8616999999999998e-05 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898903+0000", 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7623e-05 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898982+0000", 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9124999999999999e-05 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.899559+0000", 2024-09-22T01:01:55.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00057709899999999997 2024-09-22T01:01:55.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902402+0000", 2024-09-22T01:01:55.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028426990000000002 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902461+0000", 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8607999999999999e-05 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902558+0000", 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7263999999999996e-05 2024-09-22T01:01:55.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56184 3.9 3:914e0854:::rbd_data.117c93605fe7.000000000000021b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.059467764000001, 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040309569999999999, 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56184 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901364+0000", 2024-09-22T01:01:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.48e-06 2024-09-22T01:01:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901369+0000", 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1050000000000003e-06 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901384+0000", 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5098e-05 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901392+0000", 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9969999999999998e-06 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901444+0000", 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2268999999999997e-05 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901496+0000", 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1864999999999999e-05 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901562+0000", 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5718999999999999e-05 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902103+0000", 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054061300000000001 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905243+0000", 2024-09-22T01:01:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031400059999999999 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905305+0000", 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2527000000000006e-05 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905393+0000", 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8277999999999997e-05 2024-09-22T01:01:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56364 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [write 36377~4583] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.916171701, 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034458259999999999, 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56364 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044660+0000", 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.046e-06 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044672+0000", 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2119999999999999e-05 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044691+0000", 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9320000000000001e-05 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044698+0000", 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2420000000000003e-06 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044756+0000", 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7547999999999999e-05 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044836+0000", 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0657999999999999e-05 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044966+0000", 2024-09-22T01:01:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000129682 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.046764+0000", 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001797877 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.047933+0000", 2024-09-22T01:01:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001169139 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.047999+0000", 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5537000000000006e-05 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.048104+0000", 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010565699999999999 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56402 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.885619447, 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004133734, 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56402 2024-09-22T01:01:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075212+0000", 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7060000000000001e-06 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075220+0000", 2024-09-22T01:01:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2590000000000003e-06 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075235+0000", 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.537e-05 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075243+0000", 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1159999999999999e-06 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075314+0000", 2024-09-22T01:01:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0535000000000001e-05 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075399+0000", 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5745e-05 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075512+0000", 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112575 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.076225+0000", 2024-09-22T01:01:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071304799999999998 2024-09-22T01:01:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079253+0000", 2024-09-22T01:01:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030281520000000001 2024-09-22T01:01:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079290+0000", 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7240000000000003e-05 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079344+0000", 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3987999999999999e-05 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:58441 3.9 3:961db89f:::rbd_data.117c93605fe7.000000000000027c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.218959312999999, 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035419259999999999, 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 58441 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741873+0000", 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7990000000000001e-06 2024-09-22T01:01:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741879+0000", 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0249999999999999e-06 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741896+0000", 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6962000000000001e-05 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741904+0000", 2024-09-22T01:01:55.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4489999999999999e-06 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741961+0000", 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7206000000000001e-05 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742048+0000", 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7078999999999998e-05 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742156+0000", 2024-09-22T01:01:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010733799999999999 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742877+0000", 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072088000000000002 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745306+0000", 2024-09-22T01:01:55.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002429512 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745365+0000", 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8962000000000002e-05 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745413+0000", 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7713999999999998e-05 2024-09-22T01:01:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59024 3.9 3:935ab33d:::rbd_data.117ce35b6780.00000000000001db:head [set-alloc-hint object_size 262144 write_size 262144,write 33792~3072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.2432445130000005, 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048336580000000002, 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59024 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:55.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717588+0000", 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8030000000000001e-06 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717600+0000", 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2028000000000001e-05 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717626+0000", 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5930000000000001e-05 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717639+0000", 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2296000000000001e-05 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717728+0000", 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9091999999999993e-05 2024-09-22T01:01:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717864+0000", 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013604200000000001 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.718008+0000", 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014423800000000001 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.719029+0000", 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001020897 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722322+0000", 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032925509999999999 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722370+0000", 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8755000000000003e-05 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722419+0000", 2024-09-22T01:01:55.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9026000000000001e-05 2024-09-22T01:01:55.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.036 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.036 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59026 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [set-alloc-hint object_size 262144 write_size 262144,write 32768~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:55.036 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.2405754180000006, 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046681709999999996, 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59026 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720256+0000", 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3969999999999999e-06 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720267+0000", 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699e-05 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720285+0000", 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8561999999999999e-05 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720296+0000", 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0324e-05 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720362+0000", 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6156999999999997e-05 2024-09-22T01:01:55.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720474+0000", 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111656 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720641+0000", 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167875 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.721449+0000", 2024-09-22T01:01:55.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00080787999999999997 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724779+0000", 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033294090000000002 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724864+0000", 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5535e-05 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724923+0000", 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8677000000000001e-05 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59035 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.2216374479999992, 2024-09-22T01:01:55.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056261260000000004, 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59035 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739194+0000", 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7599999999999996e-07 2024-09-22T01:01:55.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739199+0000", 2024-09-22T01:01:55.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1919999999999996e-06 2024-09-22T01:01:55.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739212+0000", 2024-09-22T01:01:55.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3484e-05 2024-09-22T01:01:55.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739222+0000", 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3279999999999995e-06 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739864+0000", 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064251300000000005 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739973+0000", 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010924700000000001 2024-09-22T01:01:55.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740123+0000", 2024-09-22T01:01:55.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014958199999999999 2024-09-22T01:01:55.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740800+0000", 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067721600000000002 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744735+0000", 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039345889999999996 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744779+0000", 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4196999999999997e-05 2024-09-22T01:01:55.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744819+0000", 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0002000000000001e-05 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:59069 3.1a e70/67)", 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1479462760000008, 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033876940000000001, 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812885+0000", 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-06 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812930+0000", 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4731999999999998e-05 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812963+0000", 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3791999999999998e-05 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812986+0000", 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2493e-05 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.814703+0000", 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017169799999999999 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.814754+0000", 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1035999999999997e-05 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.816213+0000", 2024-09-22T01:01:55.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014592100000000001 2024-09-22T01:01:55.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.816272+0000", 2024-09-22T01:01:55.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8446999999999998e-05 2024-09-22T01:01:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59186 3.9 3:973f8b5e:::rbd_data.117ce35b6780.000000000000007d:head [set-alloc-hint object_size 262144 write_size 262144,write 131217~7023] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.9627744160000002, 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039873030000000002, 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59186 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998059+0000", 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998076+0000", 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.732e-05 2024-09-22T01:01:55.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998110+0000", 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3785999999999998e-05 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998124+0000", 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4544e-05 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998226+0000", 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101544 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998436+0000", 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021000899999999999 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998697+0000", 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000261406 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.999737+0000", 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010393519999999999 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001927+0000", 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021903420000000001 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001985+0000", 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7711999999999999e-05 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.002043+0000", 2024-09-22T01:01:55.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8477999999999998e-05 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59243 3.9 3:973f8b5e:::rbd_data.117ce35b6780.000000000000007d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.8840919270000001, 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032113020000000001, 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59243 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076738+0000", 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076741+0000", 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0769999999999999e-06 2024-09-22T01:01:55.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076751+0000", 2024-09-22T01:01:55.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3929999999999997e-06 2024-09-22T01:01:55.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076778+0000", 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7001000000000001e-05 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076790+0000", 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.183e-05 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076875+0000", 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5158e-05 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.076967+0000", 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2660999999999999e-05 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.077078+0000", 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110343 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.078598+0000", 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00152005 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079834+0000", 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001236343 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079897+0000", 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2724000000000005e-05 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.079950+0000", 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2722000000000001e-05 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:62483 3.1a e72/67 3:5bbcc4e4:::rbd_object_map.117c12a7c618:head v 72'176, mlcod=72'173)", 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:50.493332+0000", 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4674981039999997, 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033145420000000002, 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.069 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.493332+0000", 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.493332+0000", 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.493333+0000", 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0109999999999999e-06 2024-09-22T01:01:55.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.493342+0000", 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9109999999999999e-06 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.493365+0000", 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2750000000000001e-05 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.493377+0000", 2024-09-22T01:01:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2113000000000001e-05 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.493455+0000", 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8045000000000002e-05 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.493487+0000", 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1664e-05 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.496598+0000", 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003111001 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.496647+0000", 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9047e-05 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:62484 3.1a e72/67 3:5bbcc4e4:::rbd_object_map.117c12a7c618:head v 72'177, mlcod=72'173)", 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:50.494437+0000", 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4663927909999996, 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035320410000000001, 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494437+0000", 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494437+0000", 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494438+0000", 2024-09-22T01:01:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0100000000000003e-07 2024-09-22T01:01:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494445+0000", 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1130000000000003e-06 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494460+0000", 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4744e-05 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494470+0000", 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0309000000000001e-05 2024-09-22T01:01:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.496037+0000", 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015666530000000001 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.496066+0000", 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8668999999999999e-05 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.497906+0000", 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018403689999999999 2024-09-22T01:01:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.497969+0000", 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3282999999999997e-05 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:63727 3.9 3:95e8888c:::rbd_data.117c12a7c618.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:01:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.888808032, 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057114310000000003, 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 63727 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072037+0000", 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4440999999999999e-05 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072044+0000", 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1539999999999996e-06 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072059+0000", 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4936000000000001e-05 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072066+0000", 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.464e-06 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072130+0000", 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4054999999999996e-05 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072249+0000", 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011866099999999999 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.073379+0000", 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011302040000000001 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077601+0000", 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042215810000000003 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077622+0000", 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1622999999999999e-05 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077686+0000", 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4003000000000004e-05 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077734+0000", 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7308999999999997e-05 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.084 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:55.086 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 75, 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-22T01:00:18.015746+0000", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-22T01:01:54.195335+0000", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-22T01:01:26.960327+0000", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-22T01:01:39.066126+0000", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5804032, 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "nodeep-scrub", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-22T01:01:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-22T01:01:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-22T01:01:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:01:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-22T01:01:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:01:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:01:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:01:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "jewel", 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:30.699559+0000", 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-22T01:01:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-22T01:01:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:01:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:01:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:01:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:01:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:01:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:01:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:01:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:01:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:01:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:36.410605+0000", 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:01:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:01:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 2, 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 21, 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:01:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:01:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:01:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9900000095367432, 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9900000095367432, 2024-09-22T01:01:55.103 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:46.342370+0000", 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:01:55.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-22T01:01:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "75", 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:01:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 32, 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 75, 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:01:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:01:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:01:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.7400000095367432, 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.7400000095367432, 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:01:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "8c0f94a8-3c63-442d-9dc1-50c1278929da", 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:01:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2277223948", 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2277223948", 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2277223948", 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2277223948", 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "2d2cf30e-e9bd-40ba-8293-193faa02ea69", 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:01:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-22T01:01:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/463340194", 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/463340194", 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/463340194", 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/463340194", 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:01:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a3d3e330-9ea1-4a5c-811b-f285f9a3cff4", 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-22T01:01:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/387643397", 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/387643397", 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/387643397", 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/387643397", 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:01:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "239072f0-4b69-4624-ba6c-733f299208da", 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 55, 2024-09-22T01:01:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:01:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:01:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1215210764", 2024-09-22T01:01:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1215210764", 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1215210764", 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1215210764", 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fd35cb24-eb2c-4c17-abe9-ce5c34db9014", 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 0, 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0, 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:01:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 54, 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6800", 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6801", 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6802", 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6803", 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6806", 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6807", 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6804", 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6805", 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6801/2463696152", 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6803/2463696152", 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6807/2463696152", 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6805/2463696152", 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:01:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "b60a584d-5334-41e4-92d9-f6689de6b8bf", 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 15, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 25, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 52, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 26, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6824", 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6825", 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6826", 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6827", 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:01:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6830", 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6831", 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6828", 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6829", 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6825/1737475131", 2024-09-22T01:01:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6827/1737475131", 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1737475131", 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1737475131", 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "9be24894-1100-4e4e-bf1b-79e163fd97e0", 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:01:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6816", 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6817", 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6818", 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6819", 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6822", 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6823", 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6820", 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6821", 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6817/4220217922", 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6819/4220217922", 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6823/4220217922", 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6821/4220217922", 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "293cfbc3-8c2e-40b5-ab89-0a124cc8651c", 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:01:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6808", 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6809", 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6810", 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6811", 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6814", 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6815", 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:01:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6812", 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6813", 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6809/3867156106", 2024-09-22T01:01:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6811/3867156106", 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3867156106", 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3867156106", 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:01:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.556128+0000", 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.159753+0000", 2024-09-22T01:01:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.766199+0000", 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:01:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.155529+0000", 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 65536, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.226232+0000", 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.243346+0000", 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 26 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:01:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.143084+0000", 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.524369+0000", 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:01:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.161 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:55.162 INFO:tasks.thrashosds.thrasher:j is {'epoch': 75, 'fsid': '7d08b428-4185-4f0f-a0fc-f7dc1fefdce9', 'created': '2024-09-22T01:00:18.015746+0000', 'modified': '2024-09-22T01:01:54.195335+0000', 'last_up_change': '2024-09-22T01:01:26.960327+0000', 'last_in_change': '2024-09-22T01:01:39.066126+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:00:30.699559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:00:36.410605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:00:46.342370+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8c0f94a8-3c63-442d-9dc1-50c1278929da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2277223948}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2277223948}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2277223948}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2277223948}]}, 'public_addr': '172.21.15.2:6825/2277223948', 'cluster_addr': '172.21.15.2:6827/2277223948', 'heartbeat_back_addr': '172.21.15.2:6831/2277223948', 'heartbeat_front_addr': '172.21.15.2:6829/2277223948', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 463340194}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 463340194}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 463340194}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 463340194}]}, 'public_addr': '172.21.15.2:6817/463340194', 'cluster_addr': '172.21.15.2:6819/463340194', 'heartbeat_back_addr': '172.21.15.2:6823/463340194', 'heartbeat_front_addr': '172.21.15.2:6821/463340194', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 387643397}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 387643397}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 387643397}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 387643397}]}, 'public_addr': '172.21.15.2:6801/387643397', 'cluster_addr': '172.21.15.2:6803/387643397', 'heartbeat_back_addr': '172.21.15.2:6807/387643397', 'heartbeat_front_addr': '172.21.15.2:6805/387643397', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '239072f0-4b69-4624-ba6c-733f299208da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1215210764}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1215210764}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1215210764}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1215210764}]}, 'public_addr': '172.21.15.2:6809/1215210764', 'cluster_addr': '172.21.15.2:6811/1215210764', 'heartbeat_back_addr': '172.21.15.2:6815/1215210764', 'heartbeat_front_addr': '172.21.15.2:6813/1215210764', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 2463696152}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 2463696152}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 2463696152}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 2463696152}]}, 'public_addr': '172.21.15.119:6801/2463696152', 'cluster_addr': '172.21.15.119:6803/2463696152', 'heartbeat_back_addr': '172.21.15.119:6807/2463696152', 'heartbeat_front_addr': '172.21.15.119:6805/2463696152', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b60a584d-5334-41e4-92d9-f6689de6b8bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 25, 'up_from': 52, 'up_thru': 67, 'down_at': 26, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1737475131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1737475131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1737475131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1737475131}]}, 'public_addr': '172.21.15.119:6825/1737475131', 'cluster_addr': '172.21.15.119:6827/1737475131', 'heartbeat_back_addr': '172.21.15.119:6831/1737475131', 'heartbeat_front_addr': '172.21.15.119:6829/1737475131', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9be24894-1100-4e4e-bf1b-79e163fd97e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 4220217922}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 4220217922}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 4220217922}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 4220217922}]}, 'public_addr': '172.21.15.119:6817/4220217922', 'cluster_addr': '172.21.15.119:6819/4220217922', 'heartbeat_back_addr': '172.21.15.119:6823/4220217922', 'heartbeat_front_addr': '172.21.15.119:6821/4220217922', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '293cfbc3-8c2e-40b5-ab89-0a124cc8651c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3867156106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3867156106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3867156106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3867156106}]}, 'public_addr': '172.21.15.119:6809/3867156106', 'cluster_addr': '172.21.15.119:6811/3867156106', 'heartbeat_back_addr': '172.21.15.119:6815/3867156106', 'heartbeat_front_addr': '172.21.15.119:6813/3867156106', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.556128+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.159753+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.766199+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.155529+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.226232+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.243346+0000', 'dead_epoch': 26}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.143084+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.524369+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:01:55.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:01:55.163 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:55.083+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:55.164 INFO:teuthology.orchestra.run.smithi002.stdout:3226 read 0xadccd31 thru 0xaddad0f (0xdfdf bytes) 2024-09-22T01:01:55.164 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3228 punch from 0x5cf3d4b to 0x5cf589d, (0x1b52 bytes) 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3229 read 0x6161cee thru 0x6162a10 (0xd23 bytes) 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3230 punch from 0x816f1f to 0x8171e2, (0x2c3 bytes) 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3231 write 0x52698e8 thru 0x5276c24 (0xd33d bytes) 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3232 trunc from 0xbb06010 to 0xb922ebc 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3234 read 0x5fe4fc2 thru 0x5fec5a5 (0x75e4 bytes) 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3235 read 0x86853a6 thru 0x8694b7b (0xf7d6 bytes) 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3236 writesame 0xa2db4d2 thru 0xa2dcf0f data_size 0x1(0x1a3e bytes) 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3237 punch from 0x7f23872 to 0x7f2d48c, (0x9c1a bytes) 2024-09-22T01:01:55.165 INFO:teuthology.orchestra.run.smithi002.stdout:3238 write 0x1176798 thru 0x1181901 (0xb16a bytes) 2024-09-22T01:01:55.187 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:55.183+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:55.230 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:01:55.287 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:55.283+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:55.293 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:55.293 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:55.293 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:55.293 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:55.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:55.388 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:55.387+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:55.433 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:55.434 INFO:teuthology.orchestra.run.smithi002.stdout:3241 read 0x64846ad thru 0x6491735 (0xd089 bytes) 2024-09-22T01:01:55.434 INFO:teuthology.orchestra.run.smithi002.stdout:3242 read 0x8207e6e thru 0x820bcfd (0x3e90 bytes) 2024-09-22T01:01:55.434 INFO:teuthology.orchestra.run.smithi002.stdout:3246 read 0x8171b30 thru 0x81791ce (0x769f bytes) 2024-09-22T01:01:55.434 INFO:teuthology.orchestra.run.smithi002.stdout:3248 punch from 0x72f4bc to 0x73a16a, (0xacae bytes) 2024-09-22T01:01:55.437 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:55.437 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:55.437 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:55.437 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:55.437 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:55.444 INFO:teuthology.orchestra.run.smithi002.stdout:3249 write 0xbf81d13 thru 0xbf89838 (0x7b26 bytes) 2024-09-22T01:01:55.450 INFO:teuthology.orchestra.run.smithi002.stdout:3250 write 0x25bae09 thru 0x25bb411 (0x609 bytes) 2024-09-22T01:01:55.456 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:01:55.488 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:55.487+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:55.500 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:01:55.501 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:01:55.528 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-22T01:01:54.639548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640016179,"num_objects":1022,"num_object_clones":301,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":426,"num_read":90497,"num_read_kb":303941,"num_write":12347,"num_write_kb":188847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":867,"num_bytes_recovered":54550874,"num_keys_recovered":245,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9994,"ondisk_log_size":9994,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":393312,"kb_used_data":159692,"kb_used_omap":13,"kb_used_meta":231730,"kb_avail":660209568,"statfs":{"total":676457349120,"available":676054597632,"internally_reserved":0,"allocated":163524608,"data_stored":190332991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":237291764},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34630555,"num_objects":101,"num_object_clones":40,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":32,"num_read":6472,"num_read_kb":13576,"num_write":1048,"num_write_kb":7516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.471265"},"pg_stats":[{"pgid":"3.1f","version":"71'249","reported_seq":997,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.185246+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:01:50.185246+0000","last_peered":"2024-09-22T01:01:50.185246+0000","last_clean":"2024-09-22T01:01:50.185246+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:01:50.185246+0000","last_undegraded":"2024-09-22T01:01:50.185246+0000","last_fullsized":"2024-09-22T01:01:50.185246+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:21.678468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":26706881,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":839,"num_read_kb":9597,"num_write":283,"num_write_kb":7182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"71'546","reported_seq":2336,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.188258+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:01:50.188258+0000","last_peered":"2024-09-22T01:01:50.188258+0000","last_clean":"2024-09-22T01:01:50.188258+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:01:50.188258+0000","last_undegraded":"2024-09-22T01:01:50.188258+0000","last_fullsized":"2024-09-22T01:01:50.188258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:08:26.159072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":22369456,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":4805,"num_read_kb":9919,"num_write":804,"num_write_kb":6055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"72'265","reported_seq":3381,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.729922+0000","last_change":"2024-09-22T01:01:43.138890+0000","last_active":"2024-09-22T01:01:50.729922+0000","last_peered":"2024-09-22T01:01:50.729922+0000","last_clean":"2024-09-22T01:01:50.729922+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:01:50.729922+0000","last_undegraded":"2024-09-22T01:01:50.729922+0000","last_fullsized":"2024-09-22T01:01:50.729922+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:58:18.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23211417,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":3860,"num_read_kb":15213,"num_write":268,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"72'555","reported_seq":3220,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.703595+0000","last_change":"2024-09-22T01:01:43.141896+0000","last_active":"2024-09-22T01:01:50.703595+0000","last_peered":"2024-09-22T01:01:50.703595+0000","last_clean":"2024-09-22T01:01:50.703595+0000","last_became_active":"2024-09-22T01:01:43.141151+0000","last_became_peered":"2024-09-22T01:01:43.141151+0000","last_unstale":"2024-09-22T01:01:50.703595+0000","last_undegraded":"2024-09-22T01:01:50.703595+0000","last_fullsized":"2024-09-22T01:01:50.703595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:06:51.627600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44572931,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":4138,"num_read_kb":28719,"num_write":826,"num_write_kb":10395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":9399679,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"72'310","reported_seq":655,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.193079+0000","last_change":"2024-09-22T01:01:43.147132+0000","last_active":"2024-09-22T01:01:50.193079+0000","last_peered":"2024-09-22T01:01:50.193079+0000","last_clean":"2024-09-22T01:01:50.193079+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:01:50.193079+0000","last_undegraded":"2024-09-22T01:01:50.193079+0000","last_fullsized":"2024-09-22T01:01:50.193079+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:11.125622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":16309577,"num_objects":38,"num_object_clones":16,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":901,"num_read_kb":1054,"num_write":385,"num_write_kb":827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"72'184","reported_seq":2082,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.664669+0000","last_change":"2024-09-22T01:01:43.143586+0000","last_active":"2024-09-22T01:01:50.664669+0000","last_peered":"2024-09-22T01:01:50.664669+0000","last_clean":"2024-09-22T01:01:50.664669+0000","last_became_active":"2024-09-22T01:01:43.138644+0000","last_became_peered":"2024-09-22T01:01:43.138644+0000","last_unstale":"2024-09-22T01:01:50.664669+0000","last_undegraded":"2024-09-22T01:01:50.664669+0000","last_fullsized":"2024-09-22T01:01:50.664669+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:08.679564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":21303100,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":1784,"num_read_kb":2738,"num_write":225,"num_write_kb":4878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":156984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"70'166","reported_seq":3564,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.168593+0000","last_change":"2024-09-22T01:00:49.708786+0000","last_active":"2024-09-22T01:01:50.168593+0000","last_peered":"2024-09-22T01:01:50.168593+0000","last_clean":"2024-09-22T01:01:50.168593+0000","last_became_active":"2024-09-22T01:00:47.723339+0000","last_became_peered":"2024-09-22T01:00:47.723339+0000","last_unstale":"2024-09-22T01:01:50.168593+0000","last_undegraded":"2024-09-22T01:01:50.168593+0000","last_fullsized":"2024-09-22T01:01:50.168593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:34:40.934798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":22690867,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":3356,"num_read_kb":12980,"num_write":198,"num_write_kb":4757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"71'381","reported_seq":3975,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.187972+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:01:50.187972+0000","last_peered":"2024-09-22T01:01:50.187972+0000","last_clean":"2024-09-22T01:01:50.187972+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:01:50.187972+0000","last_undegraded":"2024-09-22T01:01:50.187972+0000","last_fullsized":"2024-09-22T01:01:50.187972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:16.701306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":39087437,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":4833,"num_read_kb":22504,"num_write":503,"num_write_kb":6278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"71'337","reported_seq":2901,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.185275+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:01:50.185275+0000","last_peered":"2024-09-22T01:01:50.185275+0000","last_clean":"2024-09-22T01:01:50.185275+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:01:50.185275+0000","last_undegraded":"2024-09-22T01:01:50.185275+0000","last_fullsized":"2024-09-22T01:01:50.185275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:22:57.916571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":10441584,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":14,"num_read":3584,"num_read_kb":6895,"num_write":427,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"71'176","reported_seq":417,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.182732+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:01:50.182732+0000","last_peered":"2024-09-22T01:01:50.182732+0000","last_clean":"2024-09-22T01:01:50.182732+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:01:50.182732+0000","last_undegraded":"2024-09-22T01:01:50.182732+0000","last_fullsized":"2024-09-22T01:01:50.182732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:29:46.832438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13207768,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":1359,"num_read_kb":8007,"num_write":169,"num_write_kb":11971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"72'403","reported_seq":5069,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.465117+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:01:50.465117+0000","last_peered":"2024-09-22T01:01:50.465117+0000","last_clean":"2024-09-22T01:01:50.465117+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:01:50.465117+0000","last_undegraded":"2024-09-22T01:01:50.465117+0000","last_fullsized":"2024-09-22T01:01:50.465117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:40:19.178444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":21009350,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":6657,"num_read_kb":18838,"num_write":444,"num_write_kb":1523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"71'319","reported_seq":3319,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.177741+0000","last_change":"2024-09-22T01:01:43.139393+0000","last_active":"2024-09-22T01:01:50.177741+0000","last_peered":"2024-09-22T01:01:50.177741+0000","last_clean":"2024-09-22T01:01:50.177741+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:01:50.177741+0000","last_undegraded":"2024-09-22T01:01:50.177741+0000","last_fullsized":"2024-09-22T01:01:50.177741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:39.020238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":27507875,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":3761,"num_read_kb":9446,"num_write":359,"num_write_kb":3193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.171419+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:01:50.171419+0000","last_peered":"2024-09-22T01:01:50.171419+0000","last_clean":"2024-09-22T01:01:50.171419+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:01:50.171419+0000","last_undegraded":"2024-09-22T01:01:50.171419+0000","last_fullsized":"2024-09-22T01:01:50.171419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"72'262","reported_seq":1577,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.184892+0000","last_change":"2024-09-22T01:00:49.729709+0000","last_active":"2024-09-22T01:01:50.184892+0000","last_peered":"2024-09-22T01:01:50.184892+0000","last_clean":"2024-09-22T01:01:50.184892+0000","last_became_active":"2024-09-22T01:00:47.718964+0000","last_became_peered":"2024-09-22T01:00:47.718964+0000","last_unstale":"2024-09-22T01:01:50.184892+0000","last_undegraded":"2024-09-22T01:01:50.184892+0000","last_fullsized":"2024-09-22T01:01:50.184892+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:45:31.414363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":16495891,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":10,"num_read":2944,"num_read_kb":4292,"num_write":245,"num_write_kb":1248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.167293+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:01:50.167293+0000","last_peered":"2024-09-22T01:01:50.167293+0000","last_clean":"2024-09-22T01:01:50.167293+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:01:50.167293+0000","last_undegraded":"2024-09-22T01:01:50.167293+0000","last_fullsized":"2024-09-22T01:01:50.167293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"72'363","reported_seq":1326,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.188556+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:01:50.188556+0000","last_peered":"2024-09-22T01:01:50.188556+0000","last_clean":"2024-09-22T01:01:50.188556+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:01:50.188556+0000","last_undegraded":"2024-09-22T01:01:50.188556+0000","last_fullsized":"2024-09-22T01:01:50.188556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:05:10.738627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":14510000,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":2008,"num_read_kb":6273,"num_write":520,"num_write_kb":2350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.178156+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:01:50.178156+0000","last_peered":"2024-09-22T01:01:50.178156+0000","last_clean":"2024-09-22T01:01:50.178156+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:01:50.178156+0000","last_undegraded":"2024-09-22T01:01:50.178156+0000","last_fullsized":"2024-09-22T01:01:50.178156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"71'210","reported_seq":2818,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.167513+0000","last_change":"2024-09-22T01:00:49.709328+0000","last_active":"2024-09-22T01:01:50.167513+0000","last_peered":"2024-09-22T01:01:50.167513+0000","last_clean":"2024-09-22T01:01:50.167513+0000","last_became_active":"2024-09-22T01:00:47.714621+0000","last_became_peered":"2024-09-22T01:00:47.714621+0000","last_unstale":"2024-09-22T01:01:50.167513+0000","last_undegraded":"2024-09-22T01:01:50.167513+0000","last_fullsized":"2024-09-22T01:01:50.167513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:51:01.878730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":9503474,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":3114,"num_read_kb":11636,"num_write":188,"num_write_kb":6011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.167482+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:01:50.167482+0000","last_peered":"2024-09-22T01:01:50.167482+0000","last_clean":"2024-09-22T01:01:50.167482+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:01:50.167482+0000","last_undegraded":"2024-09-22T01:01:50.167482+0000","last_fullsized":"2024-09-22T01:01:50.167482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'325","reported_seq":2639,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.182555+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:01:50.182555+0000","last_peered":"2024-09-22T01:01:50.182555+0000","last_clean":"2024-09-22T01:01:50.182555+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:01:50.182555+0000","last_undegraded":"2024-09-22T01:01:50.182555+0000","last_fullsized":"2024-09-22T01:01:50.182555+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:36:36.357396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12883272,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":4759,"num_read_kb":5281,"num_write":303,"num_write_kb":1017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"72'215","reported_seq":813,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.184981+0000","last_change":"2024-09-22T01:01:43.148315+0000","last_active":"2024-09-22T01:01:50.184981+0000","last_peered":"2024-09-22T01:01:50.184981+0000","last_clean":"2024-09-22T01:01:50.184981+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:01:50.184981+0000","last_undegraded":"2024-09-22T01:01:50.184981+0000","last_fullsized":"2024-09-22T01:01:50.184981+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:38:06.163498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":12454383,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":15,"num_read":838,"num_read_kb":6161,"num_write":258,"num_write_kb":3836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":76,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.182081+0000","last_change":"2024-09-22T01:01:27.722404+0000","last_active":"2024-09-22T01:01:50.182081+0000","last_peered":"2024-09-22T01:01:50.182081+0000","last_clean":"2024-09-22T01:01:50.182081+0000","last_became_active":"2024-09-22T01:01:27.721384+0000","last_became_peered":"2024-09-22T01:01:27.721384+0000","last_unstale":"2024-09-22T01:01:50.182081+0000","last_undegraded":"2024-09-22T01:01:50.182081+0000","last_fullsized":"2024-09-22T01:01:50.182081+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:10.574832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"71'1181","reported_seq":2174,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.167484+0000","last_change":"2024-09-22T01:00:49.709391+0000","last_active":"2024-09-22T01:01:50.167484+0000","last_peered":"2024-09-22T01:01:50.167484+0000","last_clean":"2024-09-22T01:01:50.167484+0000","last_became_active":"2024-09-22T01:00:47.714544+0000","last_became_peered":"2024-09-22T01:00:47.714544+0000","last_unstale":"2024-09-22T01:01:50.167484+0000","last_undegraded":"2024-09-22T01:01:50.167484+0000","last_fullsized":"2024-09-22T01:01:50.167484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:46:56.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":19437656,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":6928,"num_read_kb":11975,"num_write":2057,"num_write_kb":5957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.167092+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:01:50.167092+0000","last_peered":"2024-09-22T01:01:50.167092+0000","last_clean":"2024-09-22T01:01:50.167092+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:01:50.167092+0000","last_undegraded":"2024-09-22T01:01:50.167092+0000","last_fullsized":"2024-09-22T01:01:50.167092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"72'251","reported_seq":1677,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.502502+0000","last_change":"2024-09-22T01:01:43.152419+0000","last_active":"2024-09-22T01:01:50.502502+0000","last_peered":"2024-09-22T01:01:50.502502+0000","last_clean":"2024-09-22T01:01:50.502502+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:01:50.502502+0000","last_undegraded":"2024-09-22T01:01:50.502502+0000","last_fullsized":"2024-09-22T01:01:50.502502+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:42.841918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":11748439,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":1538,"num_read_kb":6510,"num_write":277,"num_write_kb":7928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.177172+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:01:50.177172+0000","last_peered":"2024-09-22T01:01:50.177172+0000","last_clean":"2024-09-22T01:01:50.177172+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:01:50.177172+0000","last_undegraded":"2024-09-22T01:01:50.177172+0000","last_fullsized":"2024-09-22T01:01:50.177172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"71'229","reported_seq":1438,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.188037+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:01:50.188037+0000","last_peered":"2024-09-22T01:01:50.188037+0000","last_clean":"2024-09-22T01:01:50.188037+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:01:50.188037+0000","last_undegraded":"2024-09-22T01:01:50.188037+0000","last_fullsized":"2024-09-22T01:01:50.188037+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:24:09.161330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":15000007,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":1316,"num_read_kb":5736,"num_write":251,"num_write_kb":11505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":75,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.182088+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:01:50.182088+0000","last_peered":"2024-09-22T01:01:50.182088+0000","last_clean":"2024-09-22T01:01:50.182088+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:01:50.182088+0000","last_undegraded":"2024-09-22T01:01:50.182088+0000","last_fullsized":"2024-09-22T01:01:50.182088+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":272,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.177204+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:01:50.177204+0000","last_peered":"2024-09-22T01:01:50.177204+0000","last_clean":"2024-09-22T01:01:50.177204+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:01:50.177204+0000","last_undegraded":"2024-09-22T01:01:50.177204+0000","last_fullsized":"2024-09-22T01:01:50.177204+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"72'236","reported_seq":3824,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.660941+0000","last_change":"2024-09-22T01:00:49.735787+0000","last_active":"2024-09-22T01:01:50.660941+0000","last_peered":"2024-09-22T01:01:50.660941+0000","last_clean":"2024-09-22T01:01:50.660941+0000","last_became_active":"2024-09-22T01:00:47.720325+0000","last_became_peered":"2024-09-22T01:00:47.720325+0000","last_unstale":"2024-09-22T01:01:50.660941+0000","last_undegraded":"2024-09-22T01:01:50.660941+0000","last_fullsized":"2024-09-22T01:01:50.660941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:15:18.620620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":23271711,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":4575,"num_read_kb":8860,"num_write":206,"num_write_kb":3779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"71'267","reported_seq":855,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.167019+0000","last_change":"2024-09-22T01:01:43.142999+0000","last_active":"2024-09-22T01:01:50.167019+0000","last_peered":"2024-09-22T01:01:50.167019+0000","last_clean":"2024-09-22T01:01:50.167019+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:01:50.167019+0000","last_undegraded":"2024-09-22T01:01:50.167019+0000","last_fullsized":"2024-09-22T01:01:50.167019+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:09:32.790796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":21573681,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":992,"num_read_kb":1761,"num_write":328,"num_write_kb":4265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"72'375","reported_seq":2667,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:51.290291+0000","last_change":"2024-09-22T01:01:43.139528+0000","last_active":"2024-09-22T01:01:51.290291+0000","last_peered":"2024-09-22T01:01:51.290291+0000","last_clean":"2024-09-22T01:01:51.290291+0000","last_became_active":"2024-09-22T01:01:43.138786+0000","last_became_peered":"2024-09-22T01:01:43.138786+0000","last_unstale":"2024-09-22T01:01:51.290291+0000","last_undegraded":"2024-09-22T01:01:51.290291+0000","last_fullsized":"2024-09-22T01:01:51.290291+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:49.329629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":19041758,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":3248,"num_read_kb":6578,"num_write":420,"num_write_kb":4533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":167229,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"72'330","reported_seq":1567,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.465772+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:01:50.465772+0000","last_peered":"2024-09-22T01:01:50.465772+0000","last_clean":"2024-09-22T01:01:50.465772+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:01:50.465772+0000","last_undegraded":"2024-09-22T01:01:50.465772+0000","last_fullsized":"2024-09-22T01:01:50.465772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:10:44.504633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":7961275,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":2935,"num_read_kb":13920,"num_write":406,"num_write_kb":6594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"71'220","reported_seq":462,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.182325+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:01:50.182325+0000","last_peered":"2024-09-22T01:01:50.182325+0000","last_clean":"2024-09-22T01:01:50.182325+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:01:50.182325+0000","last_undegraded":"2024-09-22T01:01:50.182325+0000","last_fullsized":"2024-09-22T01:01:50.182325+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:19.485489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20763449,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":2112,"num_read_kb":9930,"num_write":210,"num_write_kb":3299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"72'266","reported_seq":3733,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.473274+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:01:50.473274+0000","last_peered":"2024-09-22T01:01:50.473274+0000","last_clean":"2024-09-22T01:01:50.473274+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:01:50.473274+0000","last_undegraded":"2024-09-22T01:01:50.473274+0000","last_fullsized":"2024-09-22T01:01:50.473274+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:45.462707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":23317896,"num_objects":37,"num_object_clones":15,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":4354,"num_read_kb":21679,"num_write":258,"num_write_kb":12789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"72'233","reported_seq":2367,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.178122+0000","last_change":"2024-09-22T01:01:43.140327+0000","last_active":"2024-09-22T01:01:50.178122+0000","last_peered":"2024-09-22T01:01:50.178122+0000","last_clean":"2024-09-22T01:01:50.178122+0000","last_became_active":"2024-09-22T01:01:43.139857+0000","last_became_peered":"2024-09-22T01:01:43.139857+0000","last_unstale":"2024-09-22T01:01:50.178122+0000","last_undegraded":"2024-09-22T01:01:50.178122+0000","last_fullsized":"2024-09-22T01:01:50.178122+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:23:57.439208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":37891783,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":1895,"num_read_kb":10766,"num_write":239,"num_write_kb":11526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":2830133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"71'182","reported_seq":1130,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.434835+0000","last_change":"2024-09-22T01:01:43.142453+0000","last_active":"2024-09-22T01:01:50.434835+0000","last_peered":"2024-09-22T01:01:50.434835+0000","last_clean":"2024-09-22T01:01:50.434835+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:01:50.434835+0000","last_undegraded":"2024-09-22T01:01:50.434835+0000","last_fullsized":"2024-09-22T01:01:50.434835+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:18:06.249022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":11024008,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":1503,"num_read_kb":2462,"num_write":175,"num_write_kb":7082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"72'233","reported_seq":2248,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.738608+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:01:50.738608+0000","last_peered":"2024-09-22T01:01:50.738608+0000","last_clean":"2024-09-22T01:01:50.738608+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:01:50.738608+0000","last_undegraded":"2024-09-22T01:01:50.738608+0000","last_fullsized":"2024-09-22T01:01:50.738608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:52:16.942668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":28250199,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":2502,"num_read_kb":9109,"num_write":203,"num_write_kb":7590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"72'277","reported_seq":991,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.486503+0000","last_change":"2024-09-22T01:01:43.139843+0000","last_active":"2024-09-22T01:01:50.486503+0000","last_peered":"2024-09-22T01:01:50.486503+0000","last_clean":"2024-09-22T01:01:50.486503+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:01:50.486503+0000","last_undegraded":"2024-09-22T01:01:50.486503+0000","last_fullsized":"2024-09-22T01:01:50.486503+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:33:21.245967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":15686668,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":1280,"num_read_kb":3406,"num_write":366,"num_write_kb":1782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"72'158","reported_seq":1064,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.470815+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:01:50.470815+0000","last_peered":"2024-09-22T01:01:50.470815+0000","last_clean":"2024-09-22T01:01:50.470815+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:01:50.470815+0000","last_undegraded":"2024-09-22T01:01:50.470815+0000","last_fullsized":"2024-09-22T01:01:50.470815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:15.147185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":15523355,"num_objects":38,"num_object_clones":14,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":822,"num_read_kb":1638,"num_write":139,"num_write_kb":3836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"72'170","reported_seq":1134,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:01:50.178026+0000","last_change":"2024-09-22T01:01:43.141814+0000","last_active":"2024-09-22T01:01:50.178026+0000","last_peered":"2024-09-22T01:01:50.178026+0000","last_clean":"2024-09-22T01:01:50.178026+0000","last_became_active":"2024-09-22T01:01:43.141276+0000","last_became_peered":"2024-09-22T01:01:43.141276+0000","last_unstale":"2024-09-22T01:01:50.178026+0000","last_undegraded":"2024-09-22T01:01:50.178026+0000","last_fullsized":"2024-09-22T01:01:50.178026+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:54.727139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":14668644,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":841,"num_read_kb":9958,"num_write":246,"num_write_kb":9582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":5873425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":639425792,"num_objects":1018,"num_object_clones":301,"num_object_copies":2036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1018,"num_whiteouts":426,"num_read":90381,"num_read_kb":303841,"num_write":12186,"num_write_kb":186429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":867,"num_bytes_recovered":54550874,"num_keys_recovered":245,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":186269696,"data_stored":220380439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":9874,"ondisk_log_size":9874,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52648,"kb_used_data":17300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94319192,"statfs":{"total":96636764160,"available":96582852608,"internally_reserved":0,"allocated":17715200,"data_stored":20835849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299398,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64448,"kb_used_data":32172,"kb_used_omap":4,"kb_used_meta":32251,"kb_avail":94307392,"statfs":{"total":96636764160,"available":96570769408,"internally_reserved":0,"allocated":32944128,"data_stored":41019157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":33025614},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66408,"kb_used_data":26900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94305432,"statfs":{"total":96636764160,"available":96568762368,"internally_reserved":0,"allocated":27545600,"data_stored":32842128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542164,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59508,"kb_used_data":27480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312332,"statfs":{"total":96636764160,"available":96575827968,"internally_reserved":0,"allocated":28139520,"data_stored":30451177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51228,"kb_used_data":11720,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320612,"statfs":{"total":96636764160,"available":96584306688,"internally_reserved":0,"allocated":12001280,"data_stored":13840409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46012,"kb_used_data":18088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94325828,"statfs":{"total":96636764160,"available":96589647872,"internally_reserved":0,"allocated":18522112,"data_stored":22816367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53060,"kb_used_data":26032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94318780,"statfs":{"total":96636764160,"available":96582430720,"internally_reserved":0,"allocated":26656768,"data_stored":28527904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25473024,"data_stored":27797964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17895424,"data_stored":22644660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11173888,"data_stored":13467191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27770880,"data_stored":30514889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32112640,"data_stored":40643898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":18726912,"data_stored":22299068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":26324992,"data_stored":32076772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:01:55.530 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [0, 1, 2, 3] 2024-09-22T01:01:55.530 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.8', '0', '1', '2', '3'] 2024-09-22T01:01:55.530 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.8 0 1 2 3 2024-09-22T01:01:55.584 INFO:teuthology.orchestra.run.smithi002.stdout:3252 write 0x6c974be thru 0x6ca38f6 (0xc439 bytes) 2024-09-22T01:01:55.588 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:55.582+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:55.595 INFO:teuthology.orchestra.run.smithi002.stdout:3254 read 0x30c2aae thru 0x30c46d2 (0x1c25 bytes) 2024-09-22T01:01:55.595 INFO:teuthology.orchestra.run.smithi002.stdout:3256 read 0x9cfab9f thru 0x9d03fa6 (0x9408 bytes) 2024-09-22T01:01:55.596 INFO:teuthology.orchestra.run.smithi002.stdout:3257 read 0x60100e7 thru 0x6015e85 (0x5d9f bytes) 2024-09-22T01:01:55.596 INFO:teuthology.orchestra.run.smithi002.stdout:3258 write 0x17eb93f thru 0x17f9a85 (0xe147 bytes) 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.12247683, 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:01:55.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.122456001, 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.603 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:01:55.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:01:55.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.122427485999999, 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.611 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.122404835999999, 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.122382841, 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:01:55.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.122357777, 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.620 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:01:55.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:01:55.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:55.624 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.122335344, 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:01:55.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:01:55.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:01:55.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.122313867000001, 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.629 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.122285109, 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:01:55.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:01:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.121472117, 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:01:55.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:01:55.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.121447432, 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:01:55.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.646 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.117125897999999, 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:01:55.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:01:55.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.117097047, 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:01:55.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:01:55.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.117075322, 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:01:55.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:01:55.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.117054892000001, 2024-09-22T01:01:55.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:01:55.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.117032576, 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.664 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:01:55.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:01:55.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.116964781, 2024-09-22T01:01:55.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:01:55.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:01:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.116931643999999, 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:01:55.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:01:55.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.116907722000001, 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:01:55.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.681 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.116884125, 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.682 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:01:55.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:55.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:55.686 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:55.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:01:55.730 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:55.726+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:55.790 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:55.787+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:55.832 INFO:teuthology.orchestra.run.smithi002.stdout:3259 read 0x631bfd1 thru 0x631d7e3 (0x1813 bytes) 2024-09-22T01:01:55.832 INFO:teuthology.orchestra.run.smithi002.stdout:3260 read 0x988b3a5 thru 0x988ef29 (0x3b85 bytes) 2024-09-22T01:01:55.832 INFO:teuthology.orchestra.run.smithi002.stdout:3261 write 0x16f90e8 thru 0x1701232 (0x814b bytes) 2024-09-22T01:01:55.835 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:55.835 INFO:teuthology.orchestra.run.smithi002.stdout:3263 read 0x1a88677 thru 0x1a8e48f (0x5e19 bytes) 2024-09-22T01:01:55.849 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:55.849 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:3264 writesame 0xea5d270 thru 0xea5d4cc data_size 0x5(0x25d bytes) 2024-09-22T01:01:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:3266 punch from 0xab264a9 to 0xab34d81, (0xe8d8 bytes) 2024-09-22T01:01:55.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:01:55.877 INFO:teuthology.orchestra.run.smithi002.stdout:3267 trunc from 0xea5d4cd to 0x5d0918 2024-09-22T01:01:55.904 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:55.903+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:55.991 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:55.987+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:56.034 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:56.034 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:56.034 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:01:56.034 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:01:56.034 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:56.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:01:56.074 INFO:teuthology.orchestra.run.smithi002.stdout:3268 read 0x523c12 thru 0x531a06 (0xddf5 bytes) 2024-09-22T01:01:56.074 INFO:teuthology.orchestra.run.smithi002.stdout:3269 writesame 0x417186 thru 0x41817c data_size 0x1(0xff7 bytes) 2024-09-22T01:01:56.078 INFO:teuthology.orchestra.run.smithi002.stdout:3270 read 0x459800 thru 0x4680e8 (0xe8e9 bytes) 2024-09-22T01:01:56.079 INFO:teuthology.orchestra.run.smithi002.stdout:3271 punch from 0x32f189 to 0x3341de, (0x5055 bytes) 2024-09-22T01:01:56.093 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:56.086+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:56.120 INFO:teuthology.orchestra.run.smithi002.stdout:3272 write 0xea570a6 thru 0xea61168 (0xa0c3 bytes) 2024-09-22T01:01:56.127 INFO:teuthology.orchestra.run.smithi002.stdout:3274 write 0xe2868eb thru 0xe294604 (0xdd1a bytes) 2024-09-22T01:01:56.132 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.132 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.132 INFO:teuthology.orchestra.run.smithi002.stdout:3277 writesame 0xe4606dd thru 0xe4682ec data_size 0xa(0x7c10 bytes) 2024-09-22T01:01:56.141 INFO:teuthology.orchestra.run.smithi002.stdout:3278 punch from 0x7ccbb to 0x88cbc, (0xc001 bytes) 2024-09-22T01:01:56.141 INFO:teuthology.orchestra.run.smithi002.stdout:3279 read 0xcd28588 thru 0xcd2e4f8 (0x5f71 bytes) 2024-09-22T01:01:56.141 INFO:teuthology.orchestra.run.smithi002.stdout:3280 writesame 0xc2df6f3 thru 0xc2e765d data_size 0x3(0x7f6b bytes) 2024-09-22T01:01:56.157 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:01:56.158 INFO:teuthology.orchestra.run.smithi002.stdout:3283 writesame 0x34f707d thru 0x35030c3 data_size 0x1(0xc047 bytes) 2024-09-22T01:01:56.189 INFO:teuthology.orchestra.run.smithi002.stdout:3284 writesame 0x146a095 thru 0x146eb54 data_size 0x5c(0x4ac0 bytes) 2024-09-22T01:01:56.194 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:56.191+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:56.194 INFO:teuthology.orchestra.run.smithi002.stdout:3285 write 0x8f770cb thru 0x8f7d2c1 (0x61f7 bytes) 2024-09-22T01:01:56.200 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.200 INFO:teuthology.orchestra.run.smithi002.stdout:3287 read 0x3090fb4 thru 0x30a0cc9 (0xfd16 bytes) 2024-09-22T01:01:56.200 INFO:teuthology.orchestra.run.smithi002.stdout:3289 read 0x336cb3c thru 0x33778d6 (0xad9b bytes) 2024-09-22T01:01:56.200 INFO:teuthology.orchestra.run.smithi002.stdout:3290 punch from 0xd75eb9d to 0xd76a4e0, (0xb943 bytes) 2024-09-22T01:01:56.200 INFO:teuthology.orchestra.run.smithi002.stdout:3291 punch from 0x6115a8 to 0x617396, (0x5dee bytes) 2024-09-22T01:01:56.200 INFO:teuthology.orchestra.run.smithi002.stdout:3292 write 0xe35c2a8 thru 0xe36b993 (0xf6ec bytes) 2024-09-22T01:01:56.205 INFO:teuthology.orchestra.run.smithi002.stdout:3294 writesame 0xb789a52 thru 0xb791c9c data_size 0x7(0x824b bytes) 2024-09-22T01:01:56.212 INFO:teuthology.orchestra.run.smithi002.stdout:3295 write 0x137e01a thru 0x138971a (0xb701 bytes) 2024-09-22T01:01:56.215 INFO:teuthology.orchestra.run.smithi002.stderr:set 3.8 pg_upmap_items mapping to [0->1,2->3] 2024-09-22T01:01:56.219 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252413216999997, 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.223 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:01:56.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252360693999996, 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:01:56.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:01:56.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252308372999998, 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:01:56.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252142599000003, 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252114171000002, 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:01:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:01:56.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.243 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252086362, 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:56.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:01:56.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.247 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252058257000002, 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.251 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252030714999997, 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.255 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.252003066999997, 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:56.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:01:56.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:01:56.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251975922, 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:01:56.259 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:01:56.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251947962999999, 2024-09-22T01:01:56.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251920148000004, 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251893203000002, 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.274 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251865502000001, 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:01:56.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251751077999998, 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:01:56.278 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:01:56.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251711720000003, 2024-09-22T01:01:56.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:01:56.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251681564999998, 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:56.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251652307000001, 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.290 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:01:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251624431000003, 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:01:56.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.251595057000003, 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:01:56.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:01:56.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:01:56.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:01:56.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:01:56.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:01:56.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.302 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:01:56.302 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:01:56.302 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:56.303 INFO:teuthology.orchestra.run.smithi002.stdout:3296 trunc from 0xea61169 to 0x35a742f 2024-09-22T01:01:56.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:01:56.304 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:56.299+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:56.306 INFO:teuthology.orchestra.run.smithi002.stdout:3298 writesame 0xccd1f29 thru 0xccd2e1c data_size 0x1d(0xef4 bytes) 2024-09-22T01:01:56.306 INFO:teuthology.orchestra.run.smithi002.stdout:3299 write 0xc2167bb thru 0xc222a4a (0xc290 bytes) 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:3300 writesame 0x4972872 thru 0x49729b7 data_size 0x1(0x146 bytes) 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:3301 read 0x26874d9 thru 0x268a7c4 (0x32ec bytes) 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:3302 read 0x5a51515 thru 0x5a56043 (0x4b2f bytes) 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:3304 write 0x125acbb thru 0x12665fd (0xb943 bytes) 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:3307 punch from 0x70321ff to 0x7032205, (0x6 bytes) 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:3308 write 0x9dfb308 thru 0x9e016f3 (0x63ec bytes) 2024-09-22T01:01:56.307 INFO:teuthology.orchestra.run.smithi002.stdout:3309 write 0xbddb5e6 thru 0xbddecc7 (0x36e2 bytes) 2024-09-22T01:01:56.308 INFO:teuthology.orchestra.run.smithi002.stdout:3310 read 0xc0004f4 thru 0xc00c868 (0xc375 bytes) 2024-09-22T01:01:56.308 INFO:teuthology.orchestra.run.smithi002.stdout:3311 read 0x55e67f2 thru 0x55eb4c1 (0x4cd0 bytes) 2024-09-22T01:01:56.308 INFO:teuthology.orchestra.run.smithi002.stdout:3314 write 0x432cea3 thru 0x4334057 (0x71b5 bytes) 2024-09-22T01:01:56.312 INFO:teuthology.orchestra.run.smithi002.stdout:3315 write 0x8fa0513 thru 0x8fa71c4 (0x6cb2 bytes) 2024-09-22T01:01:56.325 INFO:teuthology.orchestra.run.smithi002.stdout:3316 trunc from 0xccd2e1d to 0x57ad756 2024-09-22T01:01:56.333 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:56.333 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:56.341 INFO:teuthology.orchestra.run.smithi002.stdout:3317 flatten 2024-09-22T01:01:56.348 INFO:teuthology.orchestra.run.smithi002.stdout:3318 trunc from 0x57ad756 to 0x520fd07 2024-09-22T01:01:56.352 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:56.352 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:56.358 INFO:teuthology.orchestra.run.smithi002.stdout:3319 read 0x864b1c thru 0x86e48e (0x9973 bytes) 2024-09-22T01:01:56.359 INFO:teuthology.orchestra.run.smithi002.stdout:3320 write 0x3b85ace thru 0x3b88e60 (0x3393 bytes) 2024-09-22T01:01:56.363 INFO:teuthology.orchestra.run.smithi002.stdout:3322 punch from 0x2518412 to 0x252546b, (0xd059 bytes) 2024-09-22T01:01:56.371 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:56.371 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:56.377 INFO:teuthology.orchestra.run.smithi002.stdout:3323 write 0xbd1dcf2 thru 0xbd28016 (0xa325 bytes) 2024-09-22T01:01:56.381 INFO:teuthology.orchestra.run.smithi002.stdout:3324 write 0xa99577f thru 0xa9a4250 (0xead2 bytes) 2024-09-22T01:01:56.385 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.385 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.389 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:56.389 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:56.391 INFO:teuthology.orchestra.run.smithi002.stdout:3327 writesame 0xbe9c286 thru 0xbea16d2 data_size 0x1(0x544d bytes) 2024-09-22T01:01:56.398 INFO:teuthology.orchestra.run.smithi002.stdout:3328 writesame 0xce317e8 thru 0xce337f6 data_size 0x1(0x200f bytes) 2024-09-22T01:01:56.398 INFO:teuthology.orchestra.run.smithi002.stdout:3329 read 0x960031d thru 0x960b4b5 (0xb199 bytes) 2024-09-22T01:01:56.398 INFO:teuthology.orchestra.run.smithi002.stdout:3330 read 0x326c56b thru 0x3274b85 (0x861b bytes) 2024-09-22T01:01:56.398 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.399 INFO:teuthology.orchestra.run.smithi002.stdout:3334 punch from 0x4d8af55 to 0x4d95093, (0xa13e bytes) 2024-09-22T01:01:56.403 INFO:teuthology.orchestra.run.smithi002.stdout:3335 trunc from 0xce337f7 to 0x8715a77 2024-09-22T01:01:56.404 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:56.399+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:56.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:56.424 INFO:teuthology.orchestra.run.smithi002.stdout:3336 writesame 0xa778f14 thru 0xa778f57 data_size 0x2(0x44 bytes) 2024-09-22T01:01:56.424 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.424 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.424 INFO:teuthology.orchestra.run.smithi002.stdout:3340 punch from 0x96a555 to 0x97701f, (0xcaca bytes) 2024-09-22T01:01:56.425 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.425 INFO:teuthology.orchestra.run.smithi002.stdout:3343 punch from 0x210dfc1 to 0x210f522, (0x1561 bytes) 2024-09-22T01:01:56.425 INFO:teuthology.orchestra.run.smithi002.stdout:3345 punch from 0x5e7940 to 0x5ef37b, (0x7a3b bytes) 2024-09-22T01:01:56.433 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:56.433 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:56.434 INFO:teuthology.orchestra.run.smithi002.stdout:3346 writesame 0xcadcf10 thru 0xcae6f43 data_size 0x2(0xa034 bytes) 2024-09-22T01:01:56.434 INFO:teuthology.orchestra.run.smithi002.stdout:3347 read 0xbd3445e thru 0xbd3fcd6 (0xb879 bytes) 2024-09-22T01:01:56.434 INFO:teuthology.orchestra.run.smithi002.stdout:3349 trunc from 0xcae6f44 to 0xda42932 2024-09-22T01:01:56.454 INFO:teuthology.orchestra.run.smithi002.stdout:3350 trunc from 0xda42932 to 0x1beabc9 2024-09-22T01:01:56.488 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:01:56.488 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:01:56.488 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:01:56.488 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:01:56.493 INFO:teuthology.orchestra.run.smithi002.stdout:3352 write 0x7f758b0 thru 0x7f830ae (0xd7ff bytes) 2024-09-22T01:01:56.499 INFO:teuthology.orchestra.run.smithi002.stdout:3353 read 0x39aeaf7 thru 0x39bccb8 (0xe1c2 bytes) 2024-09-22T01:01:56.502 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:01:56.520 INFO:teuthology.orchestra.run.smithi002.stdout:3354 write 0xdc3456a thru 0xdc44521 (0xffb8 bytes) 2024-09-22T01:01:56.544 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:56.543+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:56.545 INFO:teuthology.orchestra.run.smithi002.stdout:3355 trunc from 0xdc44522 to 0x3135fd1 2024-09-22T01:01:56.560 INFO:teuthology.orchestra.run.smithi002.stdout:3356 punch from 0x2a7695c to 0x2a7aa70, (0x4114 bytes) 2024-09-22T01:01:56.570 INFO:teuthology.orchestra.run.smithi002.stdout:3357 writesame 0x3599b3d thru 0x35a3be7 data_size 0x1(0xa0ab bytes) 2024-09-22T01:01:56.590 INFO:teuthology.orchestra.run.smithi002.stdout:3358 writesame 0x7977f4f thru 0x79865a6 data_size 0x6(0xe658 bytes) 2024-09-22T01:01:56.605 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:56.603+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:56.615 INFO:teuthology.orchestra.run.smithi002.stdout:3359 write 0xe5d738c thru 0xe5da98a (0x35ff bytes) 2024-09-22T01:01:56.637 INFO:teuthology.orchestra.run.smithi002.stdout:3360 trunc from 0xe5da98b to 0x79ed78b 2024-09-22T01:01:56.658 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:56.658 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:56.662 INFO:teuthology.orchestra.run.smithi002.stdout:3361 punch from 0x393b189 to 0x393e8df, (0x3756 bytes) 2024-09-22T01:01:56.673 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:56.674 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:56.678 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:56.679 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:56.687 INFO:teuthology.orchestra.run.smithi002.stdout:3362 write 0xe93b062 thru 0xe942a0f (0x79ae bytes) 2024-09-22T01:01:56.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:01:56.692 INFO:teuthology.orchestra.run.smithi002.stdout:3363 write 0x41a6e82 thru 0x41a9dca (0x2f49 bytes) 2024-09-22T01:01:56.696 INFO:teuthology.orchestra.run.smithi002.stdout:3364 punch from 0x61ee980 to 0x61fb0af, (0xc72f bytes) 2024-09-22T01:01:56.696 INFO:teuthology.orchestra.run.smithi002.stdout:3365 writesame 0x1b945e3 thru 0x1b9c57e data_size 0x2(0x7f9c bytes) 2024-09-22T01:01:56.715 INFO:teuthology.orchestra.run.smithi002.stdout:3366 trunc from 0xe942a10 to 0x9c4290d 2024-09-22T01:01:56.730 INFO:teuthology.orchestra.run.smithi002.stdout:3367 read 0x476bf9a thru 0x476c655 (0x6bc bytes) 2024-09-22T01:01:56.732 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:56.731+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:56.735 INFO:teuthology.orchestra.run.smithi002.stdout:3368 trunc from 0x9c4290d to 0x74d956e 2024-09-22T01:01:56.743 INFO:teuthology.orchestra.run.smithi002.stdout:3369 writesame 0x658e5c6 thru 0x6596270 data_size 0x5(0x7cab bytes) 2024-09-22T01:01:56.751 INFO:teuthology.orchestra.run.smithi002.stdout:3370 read 0x24e0102 thru 0x24e1d11 (0x1c10 bytes) 2024-09-22T01:01:56.752 INFO:teuthology.orchestra.run.smithi002.stdout:3371 read 0x4b315a4 thru 0x4b3733f (0x5d9c bytes) 2024-09-22T01:01:56.779 INFO:teuthology.orchestra.run.smithi002.stdout:3372 write 0xde10dfc thru 0xde1c400 (0xb605 bytes) 2024-09-22T01:01:56.785 INFO:teuthology.orchestra.run.smithi002.stdout:3373 writesame 0x32cd020 thru 0x32d5953 data_size 0x6(0x8934 bytes) 2024-09-22T01:01:56.798 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.798 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.798 INFO:teuthology.orchestra.run.smithi002.stdout:3380 read 0x57f8be3 thru 0x58074a0 (0xe8be bytes) 2024-09-22T01:01:56.798 INFO:teuthology.orchestra.run.smithi002.stdout:3381 read 0x7d0aa4b thru 0x7d0c082 (0x1638 bytes) 2024-09-22T01:01:56.799 INFO:teuthology.orchestra.run.smithi002.stdout:3384 write 0xab7f3e8 thru 0xab87fb1 (0x8bca bytes) 2024-09-22T01:01:56.805 INFO:teuthology.orchestra.run.smithi002.stdout:3387 read 0xb5b9d14 thru 0xb5c8a9a (0xed87 bytes) 2024-09-22T01:01:56.805 INFO:teuthology.orchestra.run.smithi002.stdout:3388 writesame 0x40e6d89 thru 0x40f383b data_size 0x7(0xcab3 bytes) 2024-09-22T01:01:56.807 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:56.802+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:56.818 INFO:teuthology.orchestra.run.smithi002.stdout:3391 read 0x7421121 thru 0x74267a5 (0x5685 bytes) 2024-09-22T01:01:56.819 INFO:teuthology.orchestra.run.smithi002.stdout:3393 read 0x5ead196 thru 0x5eb60ab (0x8f16 bytes) 2024-09-22T01:01:56.819 INFO:teuthology.orchestra.run.smithi002.stdout:3394 write 0x2937016 thru 0x293d4b7 (0x64a2 bytes) 2024-09-22T01:01:56.822 INFO:teuthology.orchestra.run.smithi002.stdout:3395 read 0xabb04ac thru 0xabc00d4 (0xfc29 bytes) 2024-09-22T01:01:56.822 INFO:teuthology.orchestra.run.smithi002.stdout:3396 write 0x63559b2 thru 0x635956c (0x3bbb bytes) 2024-09-22T01:01:56.830 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.830 INFO:teuthology.orchestra.run.smithi002.stdout:3398 punch from 0x5293144 to 0x5299008, (0x5ec4 bytes) 2024-09-22T01:01:56.854 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:56.854 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:56.858 INFO:teuthology.orchestra.run.smithi002.stdout:3399 trunc from 0xde1c401 to 0x36e4c11 2024-09-22T01:01:56.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:01:56.885 INFO:teuthology.orchestra.run.smithi002.stdout:3400 read 0x1382a04 thru 0x13880f8 (0x56f5 bytes) 2024-09-22T01:01:56.886 INFO:teuthology.orchestra.run.smithi002.stdout:3402 writesame 0x28e2005 thru 0x28e866b data_size 0x7(0x6667 bytes) 2024-09-22T01:01:56.893 INFO:teuthology.orchestra.run.smithi002.stdout:3403 punch from 0x162cec0 to 0x1632230, (0x5370 bytes) 2024-09-22T01:01:56.916 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:56.915+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:56.922 INFO:teuthology.orchestra.run.smithi002.stdout:3405 writesame 0x92c4923 thru 0x92cae0c data_size 0x1(0x64ea bytes) 2024-09-22T01:01:56.922 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.922 INFO:teuthology.orchestra.run.smithi002.stdout:3408 write 0xbb5dc thru 0xbb8cd (0x2f2 bytes) 2024-09-22T01:01:56.924 INFO:teuthology.orchestra.run.smithi002.stdout:3409 punch from 0x82d704d to 0x82e5703, (0xe6b6 bytes) 2024-09-22T01:01:56.924 INFO:teuthology.orchestra.run.smithi002.stdout:3410 read 0x502f71e thru 0x503e020 (0xe903 bytes) 2024-09-22T01:01:56.925 INFO:teuthology.orchestra.run.smithi002.stdout:3411 punch from 0x421bf91 to 0x422345b, (0x74ca bytes) 2024-09-22T01:01:56.925 INFO:teuthology.orchestra.run.smithi002.stdout:3412 punch from 0x22faab3 to 0x2308124, (0xd671 bytes) 2024-09-22T01:01:56.926 INFO:teuthology.orchestra.run.smithi002.stdout:3413 trunc from 0x92cae0d to 0x8ac415f 2024-09-22T01:01:56.941 INFO:teuthology.orchestra.run.smithi002.stdout:3414 punch from 0x834a581 to 0x8357396, (0xce15 bytes) 2024-09-22T01:01:56.941 INFO:teuthology.orchestra.run.smithi002.stdout:3415 read 0x64e42ea thru 0x64ea1d4 (0x5eeb bytes) 2024-09-22T01:01:56.941 INFO:teuthology.orchestra.run.smithi002.stdout:3416 punch from 0x7e93b9e to 0x7e99b66, (0x5fc8 bytes) 2024-09-22T01:01:56.956 INFO:teuthology.orchestra.run.smithi002.stdout:3417 trunc from 0x8ac415f to 0xe1d3762 2024-09-22T01:01:56.970 INFO:teuthology.orchestra.run.smithi002.stdout:3418 write 0x8d6eb63 thru 0x8d748a0 (0x5d3e bytes) 2024-09-22T01:01:56.979 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:56.979 INFO:teuthology.orchestra.run.smithi002.stdout:3421 trunc from 0xe1d3762 to 0xe1cf3b0 2024-09-22T01:01:56.993 INFO:teuthology.orchestra.run.smithi002.stdout:3422 punch from 0x5f77bfb to 0x5f7c533, (0x4938 bytes) 2024-09-22T01:01:56.995 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:56.995 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:57.010 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:01:57.010 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:57.007+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:57.011 INFO:teuthology.orchestra.run.smithi002.stdout:3423 writesame 0xe89bef2 thru 0xe8aa77a data_size 0x1(0xe889 bytes) 2024-09-22T01:01:57.011 INFO:teuthology.orchestra.run.smithi002.stdout:3424 punch from 0xc63994d to 0xc64535b, (0xba0e bytes) 2024-09-22T01:01:57.011 INFO:teuthology.orchestra.run.smithi002.stdout:3425 writesame 0x32d5bf0 thru 0x32da8de data_size 0x41(0x4cef bytes) 2024-09-22T01:01:57.014 INFO:teuthology.orchestra.run.smithi002.stdout:3426 punch from 0x8b8efe0 to 0x8b90524, (0x1544 bytes) 2024-09-22T01:01:57.014 INFO:teuthology.orchestra.run.smithi002.stdout:3427 read 0xbd452a4 thru 0xbd46980 (0x16dd bytes) 2024-09-22T01:01:57.014 INFO:teuthology.orchestra.run.smithi002.stdout:3428 read 0x187ecc4 thru 0x18850ba (0x63f7 bytes) 2024-09-22T01:01:57.015 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.036 INFO:teuthology.orchestra.run.smithi002.stdout:3430 trunc from 0xe8aa77b to 0x6863a76 2024-09-22T01:01:57.061 INFO:teuthology.orchestra.run.smithi002.stdout:3431 writesame 0x199dc8b thru 0x199e6e0 data_size 0xe(0xa56 bytes) 2024-09-22T01:01:57.063 INFO:teuthology.orchestra.run.smithi002.stdout:3432 read 0x66f4a0e thru 0x66fb1ae (0x67a1 bytes) 2024-09-22T01:01:57.063 INFO:teuthology.orchestra.run.smithi002.stdout:3435 read 0x1e07c8c thru 0x1e09c35 (0x1faa bytes) 2024-09-22T01:01:57.076 INFO:teuthology.orchestra.run.smithi002.stdout:3437 trunc from 0x6863a76 to 0xb145ada 2024-09-22T01:01:57.089 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.089 INFO:teuthology.orchestra.run.smithi002.stdout:3439 punch from 0x64c8864 to 0x64d3e99, (0xb635 bytes) 2024-09-22T01:01:57.090 INFO:teuthology.orchestra.run.smithi002.stdout:3440 punch from 0x9c1872e to 0x9c27779, (0xf04b bytes) 2024-09-22T01:01:57.090 INFO:teuthology.orchestra.run.smithi002.stdout:3441 write 0xb0f54c0 thru 0xb0fd154 (0x7c95 bytes) 2024-09-22T01:01:57.110 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:57.107+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:57.116 INFO:teuthology.orchestra.run.smithi002.stdout:3443 writesame 0xccdffa7 thru 0xcce30ec data_size 0x2(0x3146 bytes) 2024-09-22T01:01:57.116 INFO:teuthology.orchestra.run.smithi002.stdout:3444 write 0x748203b thru 0x748f154 (0xd11a bytes) 2024-09-22T01:01:57.123 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.140 INFO:teuthology.orchestra.run.smithi002.stdout:3446 write 0xdd33345 thru 0xdd41245 (0xdf01 bytes) 2024-09-22T01:01:57.148 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.149 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.149 INFO:teuthology.orchestra.run.smithi002.stdout:3450 write 0x4ffd2f3 thru 0x4ffe975 (0x1683 bytes) 2024-09-22T01:01:57.156 INFO:teuthology.orchestra.run.smithi002.stdout:3451 write 0xb0c0424 thru 0xb0cd95c (0xd539 bytes) 2024-09-22T01:01:57.160 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.164 INFO:teuthology.orchestra.run.smithi002.stdout:3456 trunc from 0xdd41246 to 0xc4f17b6 2024-09-22T01:01:57.182 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:57.182 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:57.189 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.189 INFO:teuthology.orchestra.run.smithi002.stdout:3458 read 0x9379117 thru 0x938626a (0xd154 bytes) 2024-09-22T01:01:57.189 INFO:teuthology.orchestra.run.smithi002.stdout:3459 write 0x2c2fad0 thru 0x2c31bb9 (0x20ea bytes) 2024-09-22T01:01:57.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:01:57.217 INFO:teuthology.orchestra.run.smithi002.stdout:3460 trunc from 0xc4f17b6 to 0x39d9ea0 2024-09-22T01:01:57.241 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:57.239+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:57.247 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.247 INFO:teuthology.orchestra.run.smithi002.stdout:3463 punch from 0x1a32b24 to 0x1a396cb, (0x6ba7 bytes) 2024-09-22T01:01:57.247 INFO:teuthology.orchestra.run.smithi002.stdout:3466 read 0xc5f59d thru 0xc5fdc4 (0x828 bytes) 2024-09-22T01:01:57.248 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.248 INFO:teuthology.orchestra.run.smithi002.stdout:3469 write 0x10a65e5 thru 0x10b54b6 (0xeed2 bytes) 2024-09-22T01:01:57.287 INFO:teuthology.orchestra.run.smithi002.stdout:3471 writesame 0xb0ebcd8 thru 0xb0f6ce4 data_size 0x53(0xb00d bytes) 2024-09-22T01:01:57.287 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.287 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.287 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.287 INFO:teuthology.orchestra.run.smithi002.stdout:3476 read 0x9e5baa4 thru 0x9e5d231 (0x178e bytes) 2024-09-22T01:01:57.287 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.287 INFO:teuthology.orchestra.run.smithi002.stdout:3478 write 0x47ac66d thru 0x47b7e29 (0xb7bd bytes) 2024-09-22T01:01:57.298 INFO:teuthology.orchestra.run.smithi002.stdout:3480 trunc from 0xb0f6ce5 to 0x973bfa6 2024-09-22T01:01:57.312 INFO:teuthology.orchestra.run.smithi002.stdout:3481 write 0x6f3f8e8 thru 0x6f49d5d (0xa476 bytes) 2024-09-22T01:01:57.312 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:57.311+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:57.321 INFO:teuthology.orchestra.run.smithi002.stdout:3482 write 0x50c5a60 thru 0x50c6388 (0x929 bytes) 2024-09-22T01:01:57.327 INFO:teuthology.orchestra.run.smithi002.stdout:3484 writesame 0x6abfd89 thru 0x6ac5e68 data_size 0x64(0x60e0 bytes) 2024-09-22T01:01:57.354 INFO:teuthology.orchestra.run.smithi002.stdout:3488 writesame 0xb9e4e92 thru 0xb9ebd06 data_size 0x1(0x6e75 bytes) 2024-09-22T01:01:57.354 INFO:teuthology.orchestra.run.smithi002.stdout:3489 write 0x595ee7a thru 0x596c261 (0xd3e8 bytes) 2024-09-22T01:01:57.357 INFO:teuthology.orchestra.run.smithi002.stdout:3490 write 0x7857269 thru 0x785ed4d (0x7ae5 bytes) 2024-09-22T01:01:57.367 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:57.367 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:57.367 INFO:teuthology.orchestra.run.smithi002.stdout:3491 trunc from 0xb9ebd07 to 0xa7a849c 2024-09-22T01:01:57.381 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:01:57.382 INFO:teuthology.orchestra.run.smithi002.stdout:3492 read 0x599918e thru 0x59a06b0 (0x7523 bytes) 2024-09-22T01:01:57.383 INFO:teuthology.orchestra.run.smithi002.stdout:3493 write 0x46ae3dc thru 0x46b5c21 (0x7846 bytes) 2024-09-22T01:01:57.391 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.391 INFO:teuthology.orchestra.run.smithi002.stdout:3495 read 0x3a76c27 thru 0x3a7f729 (0x8b03 bytes) 2024-09-22T01:01:57.392 INFO:teuthology.orchestra.run.smithi002.stdout:3496 writesame 0x47f7ff7 thru 0x47fe97c data_size 0xd(0x6986 bytes) 2024-09-22T01:01:57.396 INFO:teuthology.orchestra.run.smithi002.stdout:3497 write 0xa5af03 thru 0xa64e04 (0x9f02 bytes) 2024-09-22T01:01:57.404 INFO:teuthology.orchestra.run.smithi002.stdout:3498 punch from 0x74a955b to 0x74ae2c8, (0x4d6d bytes) 2024-09-22T01:01:57.404 INFO:teuthology.orchestra.run.smithi002.stdout:3500 write 0x5649266 thru 0x5655a1d (0xc7b8 bytes) 2024-09-22T01:01:57.407 INFO:teuthology.orchestra.run.smithi002.stdout:3501 writesame 0x308c1ac thru 0x3098865 data_size 0x8b(0xc6ba bytes) 2024-09-22T01:01:57.413 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:57.406+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:57.437 INFO:teuthology.orchestra.run.smithi002.stdout:3502 write 0xd1198bc thru 0xd1289bc (0xf101 bytes) 2024-09-22T01:01:57.443 INFO:teuthology.orchestra.run.smithi002.stdout:3503 write 0x88df4ad thru 0x88e6707 (0x725b bytes) 2024-09-22T01:01:57.465 INFO:teuthology.orchestra.run.smithi002.stdout:3504 write 0xdb48f79 thru 0xdb5205f (0x90e7 bytes) 2024-09-22T01:01:57.471 INFO:teuthology.orchestra.run.smithi002.stdout:3505 punch from 0x631fa20 to 0x632a801, (0xade1 bytes) 2024-09-22T01:01:57.471 INFO:teuthology.orchestra.run.smithi002.stdout:3506 write 0xce2bdb4 thru 0xce2d9b4 (0x1c01 bytes) 2024-09-22T01:01:57.477 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.477 INFO:teuthology.orchestra.run.smithi002.stdout:3508 write 0x94ec2d9 thru 0x94fbf77 (0xfc9f bytes) 2024-09-22T01:01:57.483 INFO:teuthology.orchestra.run.smithi002.stdout:3510 read 0x790faae thru 0x7916209 (0x675c bytes) 2024-09-22T01:01:57.483 INFO:teuthology.orchestra.run.smithi002.stdout:3512 write 0x9546ab2 thru 0x954edf3 (0x8342 bytes) 2024-09-22T01:01:57.490 INFO:teuthology.orchestra.run.smithi002.stdout:3513 write 0x3636146 thru 0x36375dd (0x1498 bytes) 2024-09-22T01:01:57.497 INFO:teuthology.orchestra.run.smithi002.stdout:3514 write 0x26b4c58 thru 0x26bba88 (0x6e31 bytes) 2024-09-22T01:01:57.514 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:01:57.511+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:01:57.531 INFO:teuthology.orchestra.run.smithi002.stdout:3515 trunc from 0xdb52060 to 0x23b81b9 2024-09-22T01:01:57.552 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:57.552 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:57.568 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:01:57.573 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.573 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.573 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.598 INFO:teuthology.orchestra.run.smithi002.stdout:3520 trunc from 0x23b81b9 to 0xb820ede 2024-09-22T01:01:57.612 INFO:teuthology.orchestra.run.smithi002.stdout:3521 read 0x6c03f0c thru 0x6c107d4 (0xc8c9 bytes) 2024-09-22T01:01:57.612 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.612 INFO:teuthology.orchestra.run.smithi002.stdout:3523 writesame 0x9cd175c thru 0x9cd17a1 data_size 0x2(0x46 bytes) 2024-09-22T01:01:57.615 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:57.610+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:57.619 INFO:teuthology.orchestra.run.smithi002.stdout:3524 read 0xa405db4 thru 0xa40a08e (0x42db bytes) 2024-09-22T01:01:57.633 INFO:teuthology.orchestra.run.smithi002.stdout:3525 trunc from 0xb820ede to 0x6aff986 2024-09-22T01:01:57.657 INFO:teuthology.orchestra.run.smithi002.stdout:3526 punch from 0x66705cf to 0x667338d, (0x2dbe bytes) 2024-09-22T01:01:57.665 INFO:teuthology.orchestra.run.smithi002.stdout:3527 trunc from 0x6aff986 to 0x3df4446 2024-09-22T01:01:57.680 INFO:teuthology.orchestra.run.smithi002.stdout:3529 write 0x34c2787 thru 0x34ce655 (0xbecf bytes) 2024-09-22T01:01:57.689 INFO:teuthology.orchestra.run.smithi002.stdout:3530 read 0x25c173a thru 0x25c83f6 (0x6cbd bytes) 2024-09-22T01:01:57.690 INFO:teuthology.orchestra.run.smithi002.stdout:3531 punch from 0xd614db to 0xd67a04, (0x6529 bytes) 2024-09-22T01:01:57.690 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.690 INFO:teuthology.orchestra.run.smithi002.stdout:3533 read 0x2106d08 thru 0x2109d89 (0x3082 bytes) 2024-09-22T01:01:57.690 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.690 INFO:teuthology.orchestra.run.smithi002.stdout:3536 punch from 0x1f13b38 to 0x1f2150f, (0xd9d7 bytes) 2024-09-22T01:01:57.690 INFO:teuthology.orchestra.run.smithi002.stdout:3537 read 0x2ec1977 thru 0x2ec2ae7 (0x1171 bytes) 2024-09-22T01:01:57.715 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:57.711+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:57.723 INFO:teuthology.orchestra.run.smithi002.stdout:3538 writesame 0xa4dd4f7 thru 0xa4e5a3f data_size 0x1(0x8549 bytes) 2024-09-22T01:01:57.723 INFO:teuthology.orchestra.run.smithi002.stdout:3539 read 0x10004be thru 0x1002ae3 (0x2626 bytes) 2024-09-22T01:01:57.723 INFO:teuthology.orchestra.run.smithi002.stdout:3540 writesame 0x583f6a0 thru 0x584ec54 data_size 0x57(0xf5b5 bytes) 2024-09-22T01:01:57.728 INFO:teuthology.orchestra.run.smithi002.stdout:3541 punch from 0x20b21c3 to 0x20bc7e3, (0xa620 bytes) 2024-09-22T01:01:57.735 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:57.735 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:57.740 INFO:teuthology.orchestra.run.smithi002.stdout:3542 trunc from 0xa4e5a40 to 0x5aa9b68 2024-09-22T01:01:57.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:01:57.755 INFO:teuthology.orchestra.run.smithi002.stdout:3543 read 0x1b3dcac thru 0x1b45594 (0x78e9 bytes) 2024-09-22T01:01:57.756 INFO:teuthology.orchestra.run.smithi002.stdout:3544 read 0x4f0a609 thru 0x4f189aa (0xe3a2 bytes) 2024-09-22T01:01:57.757 INFO:teuthology.orchestra.run.smithi002.stdout:3546 writesame 0x18090e2 thru 0x180b5c6 data_size 0x1(0x24e5 bytes) 2024-09-22T01:01:57.767 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.781 INFO:teuthology.orchestra.run.smithi002.stdout:3548 trunc from 0x5aa9b68 to 0xad63695 2024-09-22T01:01:57.794 INFO:teuthology.orchestra.run.smithi002.stdout:3549 read 0x19d243a thru 0x19da3f1 (0x7fb8 bytes) 2024-09-22T01:01:57.813 INFO:teuthology.orchestra.run.smithi002.stdout:3550 write 0xc5b45c9 thru 0xc5bb8cf (0x7307 bytes) 2024-09-22T01:01:57.816 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:57.810+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:57.820 INFO:teuthology.orchestra.run.smithi002.stdout:3551 read 0x34ba51e thru 0x34c1329 (0x6e0c bytes) 2024-09-22T01:01:57.822 INFO:teuthology.orchestra.run.smithi002.stdout:3552 punch from 0x2b82aab to 0x2b8cefb, (0xa450 bytes) 2024-09-22T01:01:57.822 INFO:teuthology.orchestra.run.smithi002.stdout:3554 punch from 0x4a573f2 to 0x4a5c3a4, (0x4fb2 bytes) 2024-09-22T01:01:57.822 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.822 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:57.841 INFO:teuthology.orchestra.run.smithi002.stdout:3560 trunc from 0xc5bb8d0 to 0x4c61fb2 2024-09-22T01:01:57.870 INFO:teuthology.orchestra.run.smithi002.stdout:3562 write 0x4b7d613 thru 0x4b8815a (0xab48 bytes) 2024-09-22T01:01:57.901 INFO:teuthology.orchestra.run.smithi002.stdout:3563 trunc from 0x4c61fb2 to 0xd32d7f9 2024-09-22T01:01:57.910 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:57.911 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:57.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:01:57.925 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:57.910+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:57.931 INFO:teuthology.orchestra.run.smithi002.stdout:3564 writesame 0xea54a3b thru 0xea57469 data_size 0x1(0x2a2f bytes) 2024-09-22T01:01:57.931 INFO:teuthology.orchestra.run.smithi002.stdout:3567 write 0x8073174 thru 0x80758ff (0x278c bytes) 2024-09-22T01:01:57.939 INFO:teuthology.orchestra.run.smithi002.stdout:3568 read 0xd0c2147 thru 0xd0cc967 (0xa821 bytes) 2024-09-22T01:01:57.939 INFO:teuthology.orchestra.run.smithi002.stdout:3569 write 0x60e368f thru 0x60e7e9a (0x480c bytes) 2024-09-22T01:01:57.944 INFO:teuthology.orchestra.run.smithi002.stdout:3570 punch from 0x8143aea to 0x8148d8a, (0x52a0 bytes) 2024-09-22T01:01:57.944 INFO:teuthology.orchestra.run.smithi002.stdout:3571 write 0xcb63e78 thru 0xcb6b6da (0x7863 bytes) 2024-09-22T01:01:57.971 INFO:teuthology.orchestra.run.smithi002.stdout:3572 trunc from 0xea5746a to 0x3cdec02 2024-09-22T01:01:57.990 INFO:teuthology.orchestra.run.smithi002.stdout:3573 punch from 0x1138a4d to 0x113d1aa, (0x475d bytes) 2024-09-22T01:01:57.996 INFO:teuthology.orchestra.run.smithi002.stdout:3574 trunc from 0x3cdec02 to 0x719302a 2024-09-22T01:01:58.004 INFO:teuthology.orchestra.run.smithi002.stdout:3575 read 0x1547ed3 thru 0x1556aa2 (0xebd0 bytes) 2024-09-22T01:01:58.004 INFO:teuthology.orchestra.run.smithi002.stdout:3576 write 0x5f58e98 thru 0x5f5e887 (0x59f0 bytes) 2024-09-22T01:01:58.010 INFO:teuthology.orchestra.run.smithi002.stdout:3577 read 0x195ac4d thru 0x1968a6e (0xde22 bytes) 2024-09-22T01:01:58.014 INFO:teuthology.orchestra.run.smithi002.stdout:3578 trunc from 0x719302a to 0x59a2618 2024-09-22T01:01:58.018 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:58.015+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:58.026 INFO:teuthology.orchestra.run.smithi002.stdout:3580 trunc from 0x59a2618 to 0x49b0f65 2024-09-22T01:01:58.036 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.047 INFO:teuthology.orchestra.run.smithi002.stdout:3583 trunc from 0x49b0f65 to 0xaea4692 2024-09-22T01:01:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:3584 punch from 0x7995bad to 0x79a08ef, (0xad42 bytes) 2024-09-22T01:01:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:3587 punch from 0x766f589 to 0x76720d9, (0x2b50 bytes) 2024-09-22T01:01:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:3589 read 0x7c29380 thru 0x7c332e7 (0x9f68 bytes) 2024-09-22T01:01:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:3593 writesame 0x897158f thru 0x897a542 data_size 0x4(0x8fb4 bytes) 2024-09-22T01:01:58.086 INFO:teuthology.orchestra.run.smithi002.stdout:3594 write 0xc4daf6d thru 0xc4de9ea (0x3a7e bytes) 2024-09-22T01:01:58.093 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:58.093 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:58.106 INFO:teuthology.orchestra.run.smithi002.stdout:3595 trunc from 0xc4de9eb to 0x543b285 2024-09-22T01:01:58.119 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:01:58.120 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:01:58.115+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:01:58.131 INFO:teuthology.orchestra.run.smithi002.stdout:3596 read 0x19176be thru 0x191a9fd (0x3340 bytes) 2024-09-22T01:01:58.134 INFO:teuthology.orchestra.run.smithi002.stdout:3597 trunc from 0x543b285 to 0x42a66bb 2024-09-22T01:01:58.157 INFO:teuthology.orchestra.run.smithi002.stdout:3598 write 0x571f67f thru 0x57274a0 (0x7e22 bytes) 2024-09-22T01:01:58.174 INFO:teuthology.orchestra.run.smithi002.stdout:3599 write 0x9d4e1be thru 0x9d54069 (0x5eac bytes) 2024-09-22T01:01:58.180 INFO:teuthology.orchestra.run.smithi002.stdout:3600 read 0x2e248d3 thru 0x2e25396 (0xac4 bytes) 2024-09-22T01:01:58.181 INFO:teuthology.orchestra.run.smithi002.stdout:3602 writesame 0x5092755 thru 0x509af50 data_size 0x12(0x87fc bytes) 2024-09-22T01:01:58.187 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.197 INFO:teuthology.orchestra.run.smithi002.stdout:3606 write 0xa9dd75c thru 0xa9dfbe7 (0x248c bytes) 2024-09-22T01:01:58.200 INFO:teuthology.orchestra.run.smithi002.stdout:3609 write 0x57fe27e thru 0x5802e3c (0x4bbf bytes) 2024-09-22T01:01:58.202 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.212 INFO:teuthology.orchestra.run.smithi002.stdout:3612 write 0xb24c145 thru 0xb251ec1 (0x5d7d bytes) 2024-09-22T01:01:58.219 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:58.214+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:58.220 INFO:teuthology.orchestra.run.smithi002.stdout:3613 read 0x278bd87 thru 0x27985af (0xc829 bytes) 2024-09-22T01:01:58.221 INFO:teuthology.orchestra.run.smithi002.stdout:3614 writesame 0x4a40136 thru 0x4a47e3e data_size 0x1(0x7d09 bytes) 2024-09-22T01:01:58.240 INFO:teuthology.orchestra.run.smithi002.stdout:3615 write 0x9b50981 thru 0x9b50fd6 (0x656 bytes) 2024-09-22T01:01:58.246 INFO:teuthology.orchestra.run.smithi002.stdout:3616 writesame 0x7c1521 thru 0x7d0763 data_size 0x9(0xf243 bytes) 2024-09-22T01:01:58.264 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:58.264 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:58.268 INFO:teuthology.orchestra.run.smithi002.stdout:3618 trunc from 0xb251ec2 to 0x2a379dc 2024-09-22T01:01:58.283 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:01:58.295 INFO:teuthology.orchestra.run.smithi002.stdout:3619 read 0x117fd9c thru 0x1189671 (0x98d6 bytes) 2024-09-22T01:01:58.303 INFO:teuthology.orchestra.run.smithi002.stdout:3620 trunc from 0x2a379dc to 0x74cff33 2024-09-22T01:01:58.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:01:58.312 INFO:teuthology.orchestra.run.smithi002.stdout:3621 trunc from 0x74cff33 to 0x7ee905e 2024-09-22T01:01:58.321 INFO:teuthology.orchestra.run.smithi002.stdout:3622 read 0xa12688 thru 0xa17285 (0x4bfe bytes) 2024-09-22T01:01:58.321 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:58.314+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:58.322 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.322 INFO:teuthology.orchestra.run.smithi002.stdout:3626 write 0x64d15fb thru 0x64dcc10 (0xb616 bytes) 2024-09-22T01:01:58.328 INFO:teuthology.orchestra.run.smithi002.stdout:3627 read 0x129165 thru 0x12eb2d (0x59c9 bytes) 2024-09-22T01:01:58.342 INFO:teuthology.orchestra.run.smithi002.stdout:3628 write 0xb8f3c41 thru 0xb8f76a3 (0x3a63 bytes) 2024-09-22T01:01:58.349 INFO:teuthology.orchestra.run.smithi002.stdout:3630 punch from 0xa486ebf to 0xa493fcc, (0xd10d bytes) 2024-09-22T01:01:58.349 INFO:teuthology.orchestra.run.smithi002.stdout:3631 write 0xb335849 thru 0xb343d3c (0xe4f4 bytes) 2024-09-22T01:01:58.364 INFO:teuthology.orchestra.run.smithi002.stdout:3632 write 0xcfd4811 thru 0xcfd9ee9 (0x56d9 bytes) 2024-09-22T01:01:58.370 INFO:teuthology.orchestra.run.smithi002.stdout:3633 read 0x54f7075 thru 0x54f8d9b (0x1d27 bytes) 2024-09-22T01:01:58.383 INFO:teuthology.orchestra.run.smithi002.stdout:3634 trunc from 0xcfd9eea to 0x3fdac64 2024-09-22T01:01:58.421 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:58.419+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:58.424 INFO:teuthology.orchestra.run.smithi002.stdout:3635 writesame 0xc74e217 thru 0xc75683b data_size 0x1(0x8625 bytes) 2024-09-22T01:01:58.424 INFO:teuthology.orchestra.run.smithi002.stdout:3636 punch from 0x91f49ba to 0x920236d, (0xd9b3 bytes) 2024-09-22T01:01:58.424 INFO:teuthology.orchestra.run.smithi002.stdout:3637 write 0x43adadb thru 0x43bcd50 (0xf276 bytes) 2024-09-22T01:01:58.431 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:58.431 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:58.431 INFO:teuthology.orchestra.run.smithi002.stdout:3638 read 0x9de16e3 thru 0x9de3637 (0x1f55 bytes) 2024-09-22T01:01:58.431 INFO:teuthology.orchestra.run.smithi002.stdout:3639 punch from 0x1ee1e67 to 0x1ee9b57, (0x7cf0 bytes) 2024-09-22T01:01:58.431 INFO:teuthology.orchestra.run.smithi002.stdout:3640 read 0x8382309 thru 0x83836c0 (0x13b8 bytes) 2024-09-22T01:01:58.432 INFO:teuthology.orchestra.run.smithi002.stdout:3641 write 0x2b8c17d thru 0x2b93a57 (0x78db bytes) 2024-09-22T01:01:58.433 INFO:teuthology.orchestra.run.smithi002.stdout:3642 punch from 0x35e82d7 to 0x35e85fb, (0x324 bytes) 2024-09-22T01:01:58.436 INFO:teuthology.orchestra.run.smithi002.stdout:3643 trunc from 0xc75683c to 0xa7a0a43 2024-09-22T01:01:58.451 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:01:58.453 INFO:teuthology.orchestra.run.smithi002.stdout:3644 trunc from 0xa7a0a43 to 0x59158b6 2024-09-22T01:01:58.463 INFO:teuthology.orchestra.run.smithi002.stdout:3645 writesame 0x2cce2b thru 0x2d16d1 data_size 0x1(0x48a7 bytes) 2024-09-22T01:01:58.474 INFO:teuthology.orchestra.run.smithi002.stdout:3648 read 0x16c8fd3 thru 0x16d2a77 (0x9aa5 bytes) 2024-09-22T01:01:58.481 INFO:teuthology.orchestra.run.smithi002.stdout:3649 trunc from 0x59158b6 to 0x9a033dd 2024-09-22T01:01:58.489 INFO:teuthology.orchestra.run.smithi002.stdout:3650 read 0x8f43a07 thru 0x8f52c13 (0xf20d bytes) 2024-09-22T01:01:58.489 INFO:teuthology.orchestra.run.smithi002.stdout:3651 read 0xc56b69 thru 0xc5f15f (0x85f7 bytes) 2024-09-22T01:01:58.490 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.490 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.490 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.490 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.490 INFO:teuthology.orchestra.run.smithi002.stdout:3658 writesame 0x7bb7e75 thru 0x7bb8a23 data_size 0x1(0xbaf bytes) 2024-09-22T01:01:58.497 INFO:teuthology.orchestra.run.smithi002.stdout:3659 read 0x718be1c thru 0x719451e (0x8703 bytes) 2024-09-22T01:01:58.497 INFO:teuthology.orchestra.run.smithi002.stdout:3660 read 0x122d6a1 thru 0x1231cd0 (0x4630 bytes) 2024-09-22T01:01:58.507 INFO:teuthology.orchestra.run.smithi002.stdout:3663 writesame 0xb00dc7d thru 0xb0161b4 data_size 0x74(0x8538 bytes) 2024-09-22T01:01:58.508 INFO:teuthology.orchestra.run.smithi002.stdout:3664 write 0x9f9e44a thru 0x9fa2c73 (0x482a bytes) 2024-09-22T01:01:58.513 INFO:teuthology.orchestra.run.smithi002.stdout:3665 writesame 0x687150c thru 0x6880df9 data_size 0x81(0xf8ee bytes) 2024-09-22T01:01:58.519 INFO:teuthology.orchestra.run.smithi002.stdout:3666 write 0x3f8136d thru 0x3f85196 (0x3e2a bytes) 2024-09-22T01:01:58.522 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:01:58.518+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:01:58.524 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:01:58.524 INFO:teuthology.orchestra.run.smithi002.stdout:3669 write 0x9bdb559 thru 0x9beaf19 (0xf9c1 bytes) 2024-09-22T01:01:58.543 INFO:teuthology.orchestra.run.smithi002.stdout:3670 trunc from 0xb0161b5 to 0xeaa170 2024-09-22T01:01:58.593 INFO:teuthology.orchestra.run.smithi002.stdout:3671 write 0xce69add thru 0xce72d88 (0x92ac bytes) 2024-09-22T01:01:58.597 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:58.597 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:58.600 INFO:teuthology.orchestra.run.smithi002.stdout:3672 writesame 0x818acad thru 0x8191d98 data_size 0x2c(0x70ec bytes) 2024-09-22T01:01:58.608 INFO:teuthology.orchestra.run.smithi002.stdout:3673 read 0x23168eb thru 0x2321e57 (0xb56d bytes) 2024-09-22T01:01:58.608 INFO:teuthology.orchestra.run.smithi002.stdout:3674 punch from 0x4f6961b to 0x4f696c4, (0xa9 bytes) 2024-09-22T01:01:58.608 INFO:teuthology.orchestra.run.smithi002.stdout:3676 punch from 0x364f0d7 to 0x365b51f, (0xc448 bytes) 2024-09-22T01:01:58.608 INFO:teuthology.orchestra.run.smithi002.stdout:3677 write 0x2cbff9f thru 0x2ccb446 (0xb4a8 bytes) 2024-09-22T01:01:58.610 INFO:teuthology.orchestra.run.smithi002.stdout:3678 punch from 0x4a8e21a to 0x4a93094, (0x4e7a bytes) 2024-09-22T01:01:58.611 INFO:teuthology.orchestra.run.smithi002.stdout:3679 writesame 0x9a4378b thru 0x9a5220d data_size 0x1(0xea83 bytes) 2024-09-22T01:01:58.613 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:01:58.658 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:58.654+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:58.698 INFO:teuthology.orchestra.run.smithi002.stdout:3680 write 0x9c1990f thru 0x9c1cd3c (0x342e bytes) 2024-09-22T01:01:58.701 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:01:58.708 INFO:teuthology.orchestra.run.smithi002.stdout:3682 trunc from 0xce72d89 to 0xa467236 2024-09-22T01:01:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:01:58.723 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:58.718+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:58.742 INFO:teuthology.orchestra.run.smithi002.stdout:3684 trunc from 0xa467236 to 0x2bbed92 2024-09-22T01:01:58.778 INFO:teuthology.orchestra.run.smithi002.stdout:3685 trunc from 0x2bbed92 to 0xe303f78 2024-09-22T01:01:58.796 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:01:58.796 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:01:58.805 INFO:teuthology.orchestra.run.smithi002.stdout:3686 trunc from 0xe303f78 to 0x1e13e5 2024-09-22T01:01:58.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:01:58.824 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:58.823+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:58.851 INFO:teuthology.orchestra.run.smithi002.stdout:3687 punch from 0x121f7d to 0x12eabc, (0xcb3f bytes) 2024-09-22T01:01:58.861 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:01:58.862 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:01:58.870 INFO:teuthology.orchestra.run.smithi002.stdout:3688 write 0x806fc62 thru 0x807f92c (0xfccb bytes) 2024-09-22T01:01:58.876 INFO:teuthology.orchestra.run.smithi002.stdout:3689 read 0x2fd5717 thru 0x2fdb7c0 (0x60aa bytes) 2024-09-22T01:01:58.881 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:01:58.881 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:01:58.885 INFO:teuthology.orchestra.run.smithi002.stdout:3690 trunc from 0x807f92d to 0xe2e78fe 2024-09-22T01:01:58.893 INFO:teuthology.orchestra.run.smithi002.stdout:3691 write 0xac74b0f thru 0xac7c9eb (0x7edd bytes) 2024-09-22T01:01:58.900 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:01:58.900 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:01:58.900 INFO:teuthology.orchestra.run.smithi002.stdout:3692 read 0x84f1f31 thru 0x84fb2da (0x93aa bytes) 2024-09-22T01:01:58.900 INFO:teuthology.orchestra.run.smithi002.stdout:3693 read 0x93ced1a thru 0x93db71a (0xca01 bytes) 2024-09-22T01:01:58.901 INFO:teuthology.orchestra.run.smithi002.stdout:3694 writesame 0xf3e972 thru 0xf46b6c data_size 0x37(0x81fb bytes) 2024-09-22T01:01:58.913 INFO:teuthology.orchestra.run.smithi002.stdout:3696 writesame 0xe83a302 thru 0xe8458b9 data_size 0x14(0xb5b8 bytes) 2024-09-22T01:01:58.913 INFO:teuthology.orchestra.run.smithi002.stdout:3697 read 0x477a059 thru 0x4782fef (0x8f97 bytes) 2024-09-22T01:01:58.913 INFO:teuthology.orchestra.run.smithi002.stdout:3698 read 0x8778449 thru 0x877ebb6 (0x676e bytes) 2024-09-22T01:01:58.921 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:01:58.921 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:01:58.922 INFO:teuthology.orchestra.run.smithi002.stdout:3700 write 0xea5f222 thru 0xea63755 (0x4534 bytes) 2024-09-22T01:01:58.925 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:58.923+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:58.928 INFO:teuthology.orchestra.run.smithi002.stdout:3701 clone 31 order 21 su 16384 sc 11 2024-09-22T01:01:58.943 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:01:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:01:58.964 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:01:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:01:58.979 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:58.979 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:58.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:58.979 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:58.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:58.979 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:58.979 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898470154999998, 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:58.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:01:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:01:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898451295999998, 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:58.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898433111000003, 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:01:58.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:01:58.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898415323000002, 2024-09-22T01:01:58.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:01:58.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:01:58.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898396765000001, 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898378713, 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:01:59.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:01:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:01:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:01:59.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898361364000003, 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.005 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:59.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:59.009 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898339411000002, 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:59.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:01:59.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898321875999997, 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.018 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898298089999997, 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:01:59.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898280227999997, 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:01:59.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898252991, 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:01:59.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:01:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898234285999997, 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.032 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:01:59.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.033 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:01:59.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898210788999997, 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:01:59.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898186819999999, 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.898167604999998, 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:01:59.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.897163055, 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1276 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.053 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.894709867000003, 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054655007999999998, 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1276 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083096+0000", 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083097+0000", 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083099+0000", 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2330000000000001e-06 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083106+0000", 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9439999999999999e-06 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083111+0000", 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0549999999999999e-06 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137588+0000", 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054476588999999999 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137669+0000", 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1309000000000004e-05 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137751+0000", 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2441000000000006e-05 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.894687787000002, 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.894657101999996, 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.066 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:59.067 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:01:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:01:59.068 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:01:59.068 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:01:59.069 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:01:59.069 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:59.062+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:59.168 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:59.162+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:59.240 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:59.240 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:59.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:01:59.268 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:59.267+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:59.370 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:01:59.367+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:01:59.427 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:59.427 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:59.427 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:59.427 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:59.427 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:59.442 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:01:59.486 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:59.482+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:59.565 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:59.565 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:01:59.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8964 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'20, mlcod=32'17)", 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.426040227000001, 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077894110000000004, 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:59.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136525+0000", 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3540000000000001e-06 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137650+0000", 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011257190000000001 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137671+0000", 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0560999999999998e-05 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137680+0000", 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137763+0000", 2024-09-22T01:01:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2472999999999995e-05 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137793+0000", 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-05 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144267+0000", 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064739139999999999 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144313+0000", 2024-09-22T01:01:59.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5624999999999998e-05 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8968 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'21, mlcod=32'17)", 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.421614181999999, 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004894771, 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140952+0000", 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1849999999999998e-06 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.141985+0000", 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001032984 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142001+0000", 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6776999999999999e-05 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142010+0000", 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4640000000000006e-06 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142101+0000", 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1312999999999998e-05 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142119+0000", 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8202e-05 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145810+0000", 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036909970000000001 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145844+0000", 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3849000000000001e-05 2024-09-22T01:01:59.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:9077 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'31, mlcod=32'29)", 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.167289807000003, 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034796760000000001, 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395274+0000", 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395277+0000", 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8600000000000001e-06 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395294+0000", 2024-09-22T01:01:59.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7513e-05 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395337+0000", 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2749e-05 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395358+0000", 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1097000000000001e-05 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395548+0000", 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189829 2024-09-22T01:01:59.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.395593+0000", 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4919000000000001e-05 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398634+0000", 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003041373 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.398753+0000", 2024-09-22T01:01:59.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119336 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35894 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'145, mlcod=47'143)", 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.40954893, 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055540160000000002, 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.578 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:01:59.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8700000000000002e-07 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153852+0000", 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083735000000000005 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153865+0000", 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2804e-05 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153872+0000", 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:01:59.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153943+0000", 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0908000000000006e-05 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153968+0000", 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5530000000000001e-05 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158535+0000", 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045670559999999999 2024-09-22T01:01:59.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158569+0000", 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210999999999998e-05 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37899 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'73, mlcod=48'71)", 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.383045107000001, 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012095786000000001, 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179519+0000", 2024-09-22T01:01:59.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4099999999999997e-07 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183241+0000", 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037216110000000001 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183261+0000", 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9799000000000001e-05 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183274+0000", 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3046e-05 2024-09-22T01:01:59.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183374+0000", 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010009399999999999 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183399+0000", 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5375999999999999e-05 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191540+0000", 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081403029999999998 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191614+0000", 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4715999999999994e-05 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:43527 3.9 e60/22 3:96fe8936:::rbd_data.117c83c8bd76.000000000000001b:head v 60'182, mlcod=60'180)", 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.720993896, 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005051725, 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841572+0000", 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:01:59.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845187+0000", 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036148880000000001 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845210+0000", 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2926e-05 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845221+0000", 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.17e-05 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845322+0000", 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101003 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845348+0000", 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5361e-05 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846545+0000", 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011971659999999999 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846621+0000", 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6631000000000005e-05 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56176 3.9 3:91eeadac:::rbd_data.117c93605fe7.00000000000001f7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.663809593, 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038039409999999999, 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56176 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898756+0000", 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.832e-06 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898763+0000", 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3440000000000003e-06 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898778+0000", 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4982e-05 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898787+0000", 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7479999999999999e-06 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898845+0000", 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8616999999999998e-05 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898903+0000", 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7623e-05 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898982+0000", 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9124999999999999e-05 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.899559+0000", 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00057709899999999997 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902402+0000", 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028426990000000002 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902461+0000", 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8607999999999999e-05 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902558+0000", 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7263999999999996e-05 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56184 3.9 3:914e0854:::rbd_data.117c93605fe7.000000000000021b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:59.595 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.66120115, 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040309569999999999, 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56184 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901364+0000", 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.48e-06 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901369+0000", 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1050000000000003e-06 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901384+0000", 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5098e-05 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901392+0000", 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9969999999999998e-06 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901444+0000", 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2268999999999997e-05 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901496+0000", 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1864999999999999e-05 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901562+0000", 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5718999999999999e-05 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902103+0000", 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054061300000000001 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905243+0000", 2024-09-22T01:01:59.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031400059999999999 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905305+0000", 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2527000000000006e-05 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905393+0000", 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8277999999999997e-05 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56364 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [write 36377~4583] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.517905086999999, 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034458259999999999, 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56364 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044658+0000", 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044660+0000", 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.046e-06 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044672+0000", 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2119999999999999e-05 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044691+0000", 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9320000000000001e-05 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044698+0000", 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2420000000000003e-06 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044756+0000", 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7547999999999999e-05 2024-09-22T01:01:59.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044836+0000", 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0657999999999999e-05 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.044966+0000", 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000129682 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.046764+0000", 2024-09-22T01:01:59.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001797877 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.047933+0000", 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001169139 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.047999+0000", 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5537000000000006e-05 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.048104+0000", 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010565699999999999 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56402 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.487352832999999, 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004133734, 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56402 2024-09-22T01:01:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075212+0000", 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7060000000000001e-06 2024-09-22T01:01:59.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075220+0000", 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2590000000000003e-06 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075235+0000", 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.537e-05 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075243+0000", 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1159999999999999e-06 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075314+0000", 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0535000000000001e-05 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075399+0000", 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5745e-05 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075512+0000", 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112575 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.076225+0000", 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071304799999999998 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079253+0000", 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030281520000000001 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079290+0000", 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7240000000000003e-05 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079344+0000", 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3987999999999999e-05 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:58441 3.9 3:961db89f:::rbd_data.117c93605fe7.000000000000027c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.820692699, 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035419259999999999, 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.611 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 58441 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741873+0000", 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7990000000000001e-06 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741879+0000", 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0249999999999999e-06 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741896+0000", 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6962000000000001e-05 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741904+0000", 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4489999999999999e-06 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741961+0000", 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7206000000000001e-05 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742048+0000", 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7078999999999998e-05 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742156+0000", 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010733799999999999 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742877+0000", 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072088000000000002 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745306+0000", 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002429512 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745365+0000", 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8962000000000002e-05 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745413+0000", 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7713999999999998e-05 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59024 3.9 3:935ab33d:::rbd_data.117ce35b6780.00000000000001db:head [set-alloc-hint object_size 262144 write_size 262144,write 33792~3072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:59.616 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.844977899, 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048336580000000002, 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59024 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717588+0000", 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8030000000000001e-06 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717600+0000", 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2028000000000001e-05 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717626+0000", 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5930000000000001e-05 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717639+0000", 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2296000000000001e-05 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717728+0000", 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9091999999999993e-05 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717864+0000", 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013604200000000001 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.718008+0000", 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014423800000000001 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.719029+0000", 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001020897 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722322+0000", 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032925509999999999 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722370+0000", 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8755000000000003e-05 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722419+0000", 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9026000000000001e-05 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59026 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [set-alloc-hint object_size 262144 write_size 262144,write 32768~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.842308804, 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046681709999999996, 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59026 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720256+0000", 2024-09-22T01:01:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3969999999999999e-06 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720267+0000", 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699e-05 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720285+0000", 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8561999999999999e-05 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720296+0000", 2024-09-22T01:01:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0324e-05 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720362+0000", 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6156999999999997e-05 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720474+0000", 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111656 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720641+0000", 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167875 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.721449+0000", 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00080787999999999997 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724779+0000", 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033294090000000002 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724864+0000", 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5535e-05 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724923+0000", 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8677000000000001e-05 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59035 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.823370834, 2024-09-22T01:01:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056261260000000004, 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59035 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739194+0000", 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7599999999999996e-07 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739199+0000", 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1919999999999996e-06 2024-09-22T01:01:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739212+0000", 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3484e-05 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739222+0000", 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3279999999999995e-06 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739864+0000", 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064251300000000005 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739973+0000", 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010924700000000001 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740123+0000", 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014958199999999999 2024-09-22T01:01:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740800+0000", 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067721600000000002 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744735+0000", 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039345889999999996 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744779+0000", 2024-09-22T01:01:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4196999999999997e-05 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744819+0000", 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0002000000000001e-05 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:59069 3.1a e70/67)", 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.749679662, 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033876940000000001, 2024-09-22T01:01:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812884+0000", 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812885+0000", 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-06 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812930+0000", 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4731999999999998e-05 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812963+0000", 2024-09-22T01:01:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3791999999999998e-05 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.812986+0000", 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2493e-05 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.814703+0000", 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017169799999999999 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.814754+0000", 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1035999999999997e-05 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.816213+0000", 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014592100000000001 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.816272+0000", 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8446999999999998e-05 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59186 3.9 3:973f8b5e:::rbd_data.117ce35b6780.000000000000007d:head [set-alloc-hint object_size 262144 write_size 262144,write 131217~7023] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.564507802, 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039873030000000002, 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59186 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998059+0000", 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998076+0000", 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.732e-05 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998110+0000", 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3785999999999998e-05 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998124+0000", 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4544e-05 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998226+0000", 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101544 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998436+0000", 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021000899999999999 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998697+0000", 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000261406 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.999737+0000", 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010393519999999999 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001927+0000", 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021903420000000001 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001985+0000", 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7711999999999999e-05 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.002043+0000", 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8477999999999998e-05 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:62484 3.1a e72/67 3:5bbcc4e4:::rbd_object_map.117c12a7c618:head v 72'177, mlcod=72'173)", 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:50.494437+0000", 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.0681261769999999, 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035320410000000001, 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494437+0000", 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494437+0000", 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494438+0000", 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0100000000000003e-07 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494445+0000", 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1130000000000003e-06 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494460+0000", 2024-09-22T01:01:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4744e-05 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.494470+0000", 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0309000000000001e-05 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.496037+0000", 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015666530000000001 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.496066+0000", 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8668999999999999e-05 2024-09-22T01:01:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.497906+0000", 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018403689999999999 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:50.497969+0000", 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3282999999999997e-05 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:63727 3.9 3:95e8888c:::rbd_data.117c12a7c618.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-22T01:01:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.4905414179999994, 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057114310000000003, 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 63727 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072037+0000", 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4440999999999999e-05 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072044+0000", 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1539999999999996e-06 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072059+0000", 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4936000000000001e-05 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072066+0000", 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.464e-06 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072130+0000", 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4054999999999996e-05 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072249+0000", 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011866099999999999 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.073379+0000", 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011302040000000001 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077601+0000", 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042215810000000003 2024-09-22T01:01:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077622+0000", 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1622999999999999e-05 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077686+0000", 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4003000000000004e-05 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077734+0000", 2024-09-22T01:01:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7308999999999997e-05 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:77746 3.9 3:9322efa7:::rbd_data.117c2270b223.0000000000000016:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2529512~54077] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:55.119750+0000", 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4428132329999999, 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037591759999999999, 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 77746 2024-09-22T01:01:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119750+0000", 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119750+0000", 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119752+0000", 2024-09-22T01:01:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.815e-06 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119800+0000", 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7574999999999998e-05 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119822+0000", 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2167e-05 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119834+0000", 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2269000000000001e-05 2024-09-22T01:01:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119899+0000", 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5110000000000005e-05 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.120046+0000", 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014652 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.120205+0000", 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158949 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.121429+0000", 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012244879999999999 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.123342+0000", 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019131969999999999 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.123439+0000", 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6495000000000004e-05 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.123510+0000", 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0591000000000002e-05 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:80928 3.9 3:931a72c6:::rbd_data.117c2270b223.000000000000003a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:56.245452+0000", 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.3171112119999999, 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033190630000000001, 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:01:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 80928 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245452+0000", 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245452+0000", 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245455+0000", 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4320000000000002e-06 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245463+0000", 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3920000000000003e-06 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245489+0000", 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5604000000000001e-05 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:01:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245500+0000", 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1034e-05 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245616+0000", 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011637999999999999 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245719+0000", 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102938 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.245834+0000", 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115128 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.246758+0000", 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092369499999999998 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.248670+0000", 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001912125 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.248723+0000", 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2707999999999997e-05 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:56.248771+0000", 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8627e-05 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:01:59.663 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:59.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:01:59.665 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:01:59.658+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:01:59.766 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:01:59.763+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:01:59.840 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:59.840 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:59.840 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:01:59.840 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:59.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:01:59.902 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:01:59.898+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:01:59.967 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:01:59.962+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:01:59.967 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:01:59.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:01:59.967 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:01:59.968 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:01:59.968 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:01:59.982 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:00.060 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-22T01:02:00.068 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:00.062+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:00.170 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:00.167+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:00.173 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:00.173 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:00.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698554681000001, 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698533852000001, 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.178 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698505337, 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:00.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:00.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:00.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698482686999998, 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:00.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698460692000001, 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:00.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698435627999999, 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:00.195 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698413195000001, 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698391718, 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.698362960000001, 2024-09-22T01:02:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.697549968000001, 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.697525283000001, 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.693203748999998, 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.693174897999999, 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.693153172999999, 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.693132743, 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.693110427000001, 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.693042632000001, 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.693009494999998, 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.692985573000001, 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.692961975999999, 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:00.259 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:00.260 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:00.302 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:00.298+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:00.372 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:00.366+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:00.401 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:00.401 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:00.401 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:00.401 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:00.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:00.471 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:00.471 INFO:teuthology.orchestra.run.smithi002.stdout:3703 trunc from 0xea63756 to 0xe8f4f59 2024-09-22T01:02:00.472 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:00.467+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:00.535 INFO:teuthology.orchestra.run.smithi002.stdout:3704 read 0x568ad14 thru 0x5697c4d (0xcf3a bytes) 2024-09-22T01:02:00.535 INFO:teuthology.orchestra.run.smithi002.stdout:3705 read 0xcdd2e21 thru 0xcdd6c86 (0x3e66 bytes) 2024-09-22T01:02:00.535 INFO:teuthology.orchestra.run.smithi002.stdout:3707 writesame 0x610d9d2 thru 0x6110d4f data_size 0x27(0x337e bytes) 2024-09-22T01:02:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:00.547 INFO:teuthology.orchestra.run.smithi002.stdout:3709 read 0x603e4fe thru 0x60422a8 (0x3dab bytes) 2024-09-22T01:02:00.547 INFO:teuthology.orchestra.run.smithi002.stdout:3710 write 0xd3fc6bc thru 0xd404080 (0x79c5 bytes) 2024-09-22T01:02:00.550 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:00.569 INFO:teuthology.orchestra.run.smithi002.stdout:3711 writesame 0x8657f8f thru 0x8665e7c data_size 0x41(0xdeee bytes) 2024-09-22T01:02:00.589 INFO:teuthology.orchestra.run.smithi002.stdout:3713 read 0x4924492 thru 0x492c826 (0x8395 bytes) 2024-09-22T01:02:00.590 INFO:teuthology.orchestra.run.smithi002.stdout:3714 trunc from 0xe8f4f59 to 0xe44d2e5 2024-09-22T01:02:00.593 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:00.591+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:00.604 INFO:teuthology.orchestra.run.smithi002.stdout:3716 write 0x6e3dd75 thru 0x6e484b4 (0xa740 bytes) 2024-09-22T01:02:00.626 INFO:teuthology.orchestra.run.smithi002.stdout:3719 read 0xd48137c thru 0xd48c9f5 (0xb67a bytes) 2024-09-22T01:02:00.627 INFO:teuthology.orchestra.run.smithi002.stdout:3720 read 0x296ad7b thru 0x29768e1 (0xbb67 bytes) 2024-09-22T01:02:00.627 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:00.627 INFO:teuthology.orchestra.run.smithi002.stdout:3725 read 0x9c2dc7 thru 0x9c7cb9 (0x4ef3 bytes) 2024-09-22T01:02:00.627 INFO:teuthology.orchestra.run.smithi002.stdout:3726 write 0xd2ab493 thru 0xd2b32cd (0x7e3b bytes) 2024-09-22T01:02:00.644 INFO:teuthology.orchestra.run.smithi002.stdout:3729 writesame 0x615462c thru 0x6157b83 data_size 0xc(0x3558 bytes) 2024-09-22T01:02:00.662 INFO:teuthology.orchestra.run.smithi002.stdout:3730 writesame 0xed56b3f thru 0xed58a9c data_size 0x16(0x1f5e bytes) 2024-09-22T01:02:00.665 INFO:teuthology.orchestra.run.smithi002.stdout:3731 punch from 0x3ae6ba7 to 0x3aedc79, (0x70d2 bytes) 2024-09-22T01:02:00.665 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:00.674 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:00.671+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:00.687 INFO:teuthology.orchestra.run.smithi002.stdout:3734 trunc from 0xed58a9d to 0x52a2e10 2024-09-22T01:02:00.726 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:00.726 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:00.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:00.726 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:00.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755821830000002, 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:02:00.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:02:00.730 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755769307000001, 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.731 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:02:00.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:02:00.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755716986000003, 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:02:00.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.738 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755551212, 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:02:00.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.742 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755522784, 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755494974999998, 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:02:00.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:02:00.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:02:00.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755466869999999, 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:02:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:02:00.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755439328000001, 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755411680000002, 2024-09-22T01:02:00.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755384534999997, 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:00.767 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755356575999997, 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:02:00.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755328761000001, 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:02:00.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:02:00.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.775 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755301815999999, 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755274114999999, 2024-09-22T01:02:00.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755159691000003, 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:02:00.783 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:02:00.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755120333000001, 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:02:00.787 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755090178000003, 2024-09-22T01:02:00.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755060919999998, 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755033044000001, 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:02:00.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.755003670000001, 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:02:00.803 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:02:00.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:00.808 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:00.808 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:02:00.810 INFO:teuthology.orchestra.run.smithi002.stdout:3735 write 0x7441a38 thru 0x744af28 (0x94f1 bytes) 2024-09-22T01:02:00.810 INFO:teuthology.orchestra.run.smithi002.stdout:3736 read 0x4d2f2c0 thru 0x4d35188 (0x5ec9 bytes) 2024-09-22T01:02:00.810 INFO:teuthology.orchestra.run.smithi002.stdout:3737 flatten 2024-09-22T01:02:00.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:00.812 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:00.807+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:00.873 INFO:teuthology.orchestra.run.smithi002.stdout:3738 read 0x607248f thru 0x608016b (0xdcdd bytes) 2024-09-22T01:02:00.873 INFO:teuthology.orchestra.run.smithi002.stdout:3740 read 0x1964fab thru 0x1969765 (0x47bb bytes) 2024-09-22T01:02:00.874 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:00.874 INFO:teuthology.orchestra.run.smithi002.stdout:3743 punch from 0x2267b06 to 0x226dab0, (0x5faa bytes) 2024-09-22T01:02:00.874 INFO:teuthology.orchestra.run.smithi002.stdout:3746 write 0x119bf02 thru 0x11a0243 (0x4342 bytes) 2024-09-22T01:02:00.877 INFO:teuthology.orchestra.run.smithi002.stdout:3747 write 0x4079f3f thru 0x4083518 (0x95da bytes) 2024-09-22T01:02:00.884 INFO:teuthology.orchestra.run.smithi002.stdout:3748 write 0x583a5ee thru 0x5840a14 (0x6427 bytes) 2024-09-22T01:02:00.890 INFO:teuthology.orchestra.run.smithi002.stdout:3749 write 0x103dea6 thru 0x1040ede (0x3039 bytes) 2024-09-22T01:02:00.892 INFO:teuthology.orchestra.run.smithi002.stdout:3750 punch from 0x234f2e0 to 0x235bb62, (0xc882 bytes) 2024-09-22T01:02:00.892 INFO:teuthology.orchestra.run.smithi002.stdout:3754 trunc from 0x744af29 to 0x6fb492b 2024-09-22T01:02:00.903 INFO:teuthology.orchestra.run.smithi002.stdout:3755 trunc from 0x6fb492b to 0x8e2d1e9 2024-09-22T01:02:00.910 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:00.907+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:00.912 INFO:teuthology.orchestra.run.smithi002.stdout:3756 writesame 0x818ce0a thru 0x8197985 data_size 0x32(0xab7c bytes) 2024-09-22T01:02:00.918 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:00.918 INFO:teuthology.orchestra.run.smithi002.stdout:3758 read 0x2be488d thru 0x2be76ef (0x2e63 bytes) 2024-09-22T01:02:00.933 INFO:teuthology.orchestra.run.smithi002.stdout:3759 write 0xd5773b2 thru 0xd579a9e (0x26ed bytes) 2024-09-22T01:02:00.939 INFO:teuthology.orchestra.run.smithi002.stdout:3762 write 0x21537bd thru 0x2153a9a (0x2de bytes) 2024-09-22T01:02:00.943 INFO:teuthology.orchestra.run.smithi002.stdout:3763 writesame 0x708846 thru 0x70ab91 data_size 0x12(0x234c bytes) 2024-09-22T01:02:00.945 INFO:teuthology.orchestra.run.smithi002.stdout:3764 punch from 0x24ddf2c to 0x24e2f57, (0x502b bytes) 2024-09-22T01:02:00.945 INFO:teuthology.orchestra.run.smithi002.stdout:3765 write 0x81aa2c2 thru 0x81ad87c (0x35bb bytes) 2024-09-22T01:02:00.947 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:00.947 INFO:teuthology.orchestra.run.smithi002.stdout:3770 write 0x57e1d9f thru 0x57ead00 (0x8f62 bytes) 2024-09-22T01:02:00.949 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:00.949 INFO:teuthology.orchestra.run.smithi002.stdout:3772 writesame 0xaa090fa thru 0xaa0ade3 data_size 0x1(0x1cea bytes) 2024-09-22T01:02:00.965 INFO:teuthology.orchestra.run.smithi002.stdout:3774 trunc from 0xd579a9f to 0x77ed4e3 2024-09-22T01:02:00.983 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:00.984 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:00.984 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:00.984 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:00.992 INFO:teuthology.orchestra.run.smithi002.stdout:3775 trunc from 0x77ed4e3 to 0xea5aaa9 2024-09-22T01:02:00.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:01.011 INFO:teuthology.orchestra.run.smithi002.stdout:3776 trunc from 0xea5aaa9 to 0x66f5482 2024-09-22T01:02:01.027 INFO:teuthology.orchestra.run.smithi002.stdout:3777 write 0xd9c01c thru 0xdabebb (0xfea0 bytes) 2024-09-22T01:02:01.031 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.031 INFO:teuthology.orchestra.run.smithi002.stdout:3780 punch from 0x46f7855 to 0x4704574, (0xcd1f bytes) 2024-09-22T01:02:01.031 INFO:teuthology.orchestra.run.smithi002.stdout:3781 read 0x61a2171 thru 0x61ac991 (0xa821 bytes) 2024-09-22T01:02:01.031 INFO:teuthology.orchestra.run.smithi002.stdout:3782 writesame 0x1411c3a thru 0x141db0d data_size 0x8a(0xbed4 bytes) 2024-09-22T01:02:01.034 INFO:teuthology.orchestra.run.smithi002.stdout:3783 read 0x3ae5b5a thru 0x3af300b (0xd4b2 bytes) 2024-09-22T01:02:01.040 INFO:teuthology.orchestra.run.smithi002.stdout:3784 write 0x688a5eb thru 0x6899d37 (0xf74d bytes) 2024-09-22T01:02:01.040 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:01.039+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:01.049 INFO:teuthology.orchestra.run.smithi002.stdout:3785 trunc from 0x6899d38 to 0x9257695 2024-09-22T01:02:01.056 INFO:teuthology.orchestra.run.smithi002.stdout:3788 punch from 0x5dfe7c9 to 0x5e02b86, (0x43bd bytes) 2024-09-22T01:02:01.056 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.056 INFO:teuthology.orchestra.run.smithi002.stdout:3793 write 0x6689b60 thru 0x668c620 (0x2ac1 bytes) 2024-09-22T01:02:01.060 INFO:teuthology.orchestra.run.smithi002.stdout:3795 write 0x586f363 thru 0x58704bc (0x115a bytes) 2024-09-22T01:02:01.062 INFO:teuthology.orchestra.run.smithi002.stdout:3796 writesame 0x6997bcc thru 0x69a055d data_size 0x1(0x8992 bytes) 2024-09-22T01:02:01.070 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:01.076 INFO:teuthology.orchestra.run.smithi002.stdout:3797 read 0x1485fab thru 0x1492032 (0xc088 bytes) 2024-09-22T01:02:01.076 INFO:teuthology.orchestra.run.smithi002.stdout:3798 read 0x811b592 thru 0x8128aa6 (0xd515 bytes) 2024-09-22T01:02:01.076 INFO:teuthology.orchestra.run.smithi002.stdout:3799 writesame 0xad8077 thru 0xae39e3 data_size 0x1(0xb96d bytes) 2024-09-22T01:02:01.097 INFO:teuthology.orchestra.run.smithi002.stdout:3800 writesame 0x9b3d181 thru 0x9b44c1c data_size 0x1c(0x7a9c bytes) 2024-09-22T01:02:01.097 INFO:teuthology.orchestra.run.smithi002.stdout:3802 read 0x65ea65b thru 0x65ebc06 (0x15ac bytes) 2024-09-22T01:02:01.097 INFO:teuthology.orchestra.run.smithi002.stdout:3803 read 0x641e59b thru 0x642b60b (0xd071 bytes) 2024-09-22T01:02:01.098 INFO:teuthology.orchestra.run.smithi002.stdout:3804 read 0x85879a6 thru 0x858b791 (0x3dec bytes) 2024-09-22T01:02:01.098 INFO:teuthology.orchestra.run.smithi002.stdout:3805 read 0x251432f thru 0x251b730 (0x7402 bytes) 2024-09-22T01:02:01.098 INFO:teuthology.orchestra.run.smithi002.stdout:3807 read 0x4112004 thru 0x4118df0 (0x6ded bytes) 2024-09-22T01:02:01.099 INFO:teuthology.orchestra.run.smithi002.stdout:3809 write 0x24e4d9 thru 0x255912 (0x743a bytes) 2024-09-22T01:02:01.101 INFO:teuthology.orchestra.run.smithi002.stdout:3810 write 0x1e36820 thru 0x1e37f19 (0x16fa bytes) 2024-09-22T01:02:01.106 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.107 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.107 INFO:teuthology.orchestra.run.smithi002.stdout:3814 punch from 0x117e1cb to 0x118d15d, (0xef92 bytes) 2024-09-22T01:02:01.112 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:01.106+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:01.120 INFO:teuthology.orchestra.run.smithi002.stdout:3815 writesame 0xdc84d55 thru 0xdc89dc5 data_size 0x1(0x5071 bytes) 2024-09-22T01:02:01.139 INFO:teuthology.orchestra.run.smithi002.stdout:3816 trunc from 0xdc89dc6 to 0x4434a7 2024-09-22T01:02:01.172 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:01.172 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:01.172 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:01.172 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:01.172 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:01.187 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:01.188 INFO:teuthology.orchestra.run.smithi002.stdout:3817 writesame 0x9b5bb15 thru 0x9b69189 data_size 0x4d(0xd675 bytes) 2024-09-22T01:02:01.188 INFO:teuthology.orchestra.run.smithi002.stdout:3818 write 0x8945146 thru 0x894ed32 (0x9bed bytes) 2024-09-22T01:02:01.194 INFO:teuthology.orchestra.run.smithi002.stdout:3820 write 0x634a61d thru 0x634d570 (0x2f54 bytes) 2024-09-22T01:02:01.199 INFO:teuthology.orchestra.run.smithi002.stdout:3822 read 0x79a80ec thru 0x79b7e23 (0xfd38 bytes) 2024-09-22T01:02:01.207 INFO:teuthology.orchestra.run.smithi002.stdout:3823 write 0x9f56033 thru 0x9f5aeb9 (0x4e87 bytes) 2024-09-22T01:02:01.212 INFO:teuthology.orchestra.run.smithi002.stdout:3824 read 0x6fba02 thru 0x704a95 (0x9094 bytes) 2024-09-22T01:02:01.213 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.213 INFO:teuthology.orchestra.run.smithi002.stdout:3828 write 0x70e0274 thru 0x70e9363 (0x90f0 bytes) 2024-09-22T01:02:01.225 INFO:teuthology.orchestra.run.smithi002.stdout:3829 write 0x9fea990 thru 0x9ff965b (0xeccc bytes) 2024-09-22T01:02:01.232 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:01.231+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:01.234 INFO:teuthology.orchestra.run.smithi002.stdout:3830 read 0x18020d1 thru 0x1812066 (0xff96 bytes) 2024-09-22T01:02:01.234 INFO:teuthology.orchestra.run.smithi002.stdout:3831 write 0x7ad8b76 thru 0x7add5f8 (0x4a83 bytes) 2024-09-22T01:02:01.235 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:01.236 INFO:teuthology.orchestra.run.smithi002.stdout:3832 punch from 0x34685e8 to 0x3473652, (0xb06a bytes) 2024-09-22T01:02:01.236 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.236 INFO:teuthology.orchestra.run.smithi002.stdout:3834 write 0x687801 thru 0x689f14 (0x2714 bytes) 2024-09-22T01:02:01.247 INFO:teuthology.orchestra.run.smithi002.stdout:3835 trunc from 0x9ff965c to 0x3b4c8fb 2024-09-22T01:02:01.256 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:01.256 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:01.262 INFO:teuthology.orchestra.run.smithi002.stdout:3836 punch from 0x14db570 to 0x14e65e7, (0xb077 bytes) 2024-09-22T01:02:01.263 INFO:teuthology.orchestra.run.smithi002.stdout:3837 read 0x276b3f4 thru 0x27731b1 (0x7dbe bytes) 2024-09-22T01:02:01.263 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.263 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.263 INFO:teuthology.orchestra.run.smithi002.stdout:3840 write 0x94b2e3 thru 0x95743e (0xc15c bytes) 2024-09-22T01:02:01.272 INFO:teuthology.orchestra.run.smithi002.stdout:3841 trunc from 0x3b4c8fb to 0x7ea3b5c 2024-09-22T01:02:01.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:01.280 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:01.287 INFO:teuthology.orchestra.run.smithi002.stdout:3843 write 0x9f46569 thru 0x9f4fd08 (0x97a0 bytes) 2024-09-22T01:02:01.293 INFO:teuthology.orchestra.run.smithi002.stdout:3844 read 0x4923d81 thru 0x4931e2e (0xe0ae bytes) 2024-09-22T01:02:01.294 INFO:teuthology.orchestra.run.smithi002.stdout:3845 trunc from 0x9f4fd09 to 0x94c7fc2 2024-09-22T01:02:01.299 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:01.300 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:01.306 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:02:01.306 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:02:01.306 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-22T01:02:01.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:02:01.315 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9574315970000002, 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9573946280000003, 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9573663589999999, 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:02:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9573284829999995, 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:02:01.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:02:01.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9572898930000004, 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.334 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:02:01.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:02:01.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:02:01.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9572175649999997, 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:02:01.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:01.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9571796990000001, 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:02:01.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9571374600000002, 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:02:01.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9570830749999999, 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:02:01.351 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:02:01.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9569823099999999, 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:02:01.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9568953850000002, 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:01.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:02:01.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9568370609999999, 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.365 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:02:01.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:02:01.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:02:01.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9563839109999996, 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:02:01.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:02:01.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9557454249999999, 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.373 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:02:01.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:02:01.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9554578280000001, 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.378 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:02:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:02:01.381 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9548548590000001, 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:02:01.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9537745600000003, 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:02:01.386 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:02:01.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:02:01.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9537046819999997, 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.391 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:02:01.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:02:01.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.953142691, 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.395 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:01.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:02:01.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:02:01.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.9527652660000001, 2024-09-22T01:02:01.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:02:01.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:02:01.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:01.403 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:01.404 INFO:teuthology.orchestra.run.smithi002.stdout:3846 trunc from 0x94c7fc2 to 0xecbc56b 2024-09-22T01:02:01.405 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:01.314+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:01.405 INFO:teuthology.orchestra.run.smithi002.stdout:3847 writesame 0xac96f2a thru 0xac9ec0a data_size 0x1(0x7ce1 bytes) 2024-09-22T01:02:01.405 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:01.406 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:01.406 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:01.407 INFO:teuthology.orchestra.run.smithi002.stdout:3848 read 0xac15b09 thru 0xac1cf9c (0x7494 bytes) 2024-09-22T01:02:01.407 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:01.407 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:01.408 INFO:teuthology.orchestra.run.smithi002.stdout:3850 trunc from 0xecbc56b to 0x995345e 2024-09-22T01:02:01.408 INFO:teuthology.orchestra.run.smithi002.stdout:3852 punch from 0x73f8331 to 0x73f9764, (0x1433 bytes) 2024-09-22T01:02:01.408 INFO:teuthology.orchestra.run.smithi002.stdout:3853 read 0x71560a4 thru 0x715bbb5 (0x5b12 bytes) 2024-09-22T01:02:01.408 INFO:teuthology.orchestra.run.smithi002.stdout:3854 writesame 0xe32239 thru 0xe378cd data_size 0x1(0x5695 bytes) 2024-09-22T01:02:01.408 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:01.408 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:01.409 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.409 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.409 INFO:teuthology.orchestra.run.smithi002.stdout:3857 trunc from 0x995345e to 0x8d55fd5 2024-09-22T01:02:01.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:01.410 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:01.410 INFO:teuthology.orchestra.run.smithi002.stdout:3858 writesame 0xcaa11cc thru 0xcaabc5c data_size 0x47(0xaa91 bytes) 2024-09-22T01:02:01.410 INFO:teuthology.orchestra.run.smithi002.stdout:3860 read 0xc0fa5c3 thru 0xc105c4e (0xb68c bytes) 2024-09-22T01:02:01.410 INFO:teuthology.orchestra.run.smithi002.stdout:3861 punch from 0x4dadc71 to 0x4db9b50, (0xbedf bytes) 2024-09-22T01:02:01.410 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.411 INFO:teuthology.orchestra.run.smithi002.stdout:3866 writesame 0xc35a004 thru 0xc36594f data_size 0x4(0xb94c bytes) 2024-09-22T01:02:01.411 INFO:teuthology.orchestra.run.smithi002.stdout:3867 punch from 0xabfc9c3 to 0xac0c392, (0xf9cf bytes) 2024-09-22T01:02:01.411 INFO:teuthology.orchestra.run.smithi002.stdout:3869 writesame 0xad558e thru 0xadb096 data_size 0x5(0x5b09 bytes) 2024-09-22T01:02:01.413 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:01.406+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:01.415 INFO:teuthology.orchestra.run.smithi002.stdout:3870 write 0x73a9ea2 thru 0x73afe72 (0x5fd1 bytes) 2024-09-22T01:02:01.420 INFO:teuthology.orchestra.run.smithi002.stdout:3872 read 0x801eb54 thru 0x802685e (0x7d0b bytes) 2024-09-22T01:02:01.420 INFO:teuthology.orchestra.run.smithi002.stdout:3873 read 0x1f74ba4 thru 0x1f8000a (0xb467 bytes) 2024-09-22T01:02:01.420 INFO:teuthology.orchestra.run.smithi002.stdout:3874 write 0x162a192 thru 0x162eb0b (0x497a bytes) 2024-09-22T01:02:01.435 INFO:teuthology.orchestra.run.smithi002.stdout:3875 trunc from 0xcaabc5d to 0x61bb332 2024-09-22T01:02:01.450 INFO:teuthology.orchestra.run.smithi002.stdout:3877 punch from 0x5ce359b to 0x5ced72f, (0xa194 bytes) 2024-09-22T01:02:01.458 INFO:teuthology.orchestra.run.smithi002.stdout:3878 writesame 0x6e70383 thru 0x6e7e8e9 data_size 0x1(0xe567 bytes) 2024-09-22T01:02:01.474 INFO:teuthology.orchestra.run.smithi002.stdout:3880 write 0xdcc9c13 thru 0xdccd274 (0x3662 bytes) 2024-09-22T01:02:01.479 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.479 INFO:teuthology.orchestra.run.smithi002.stdout:3882 read 0x45261e7 thru 0x452740b (0x1225 bytes) 2024-09-22T01:02:01.479 INFO:teuthology.orchestra.run.smithi002.stdout:3883 read 0xb795140 thru 0xb79e3d1 (0x9292 bytes) 2024-09-22T01:02:01.479 INFO:teuthology.orchestra.run.smithi002.stdout:3884 writesame 0x3d36de9 thru 0x3d45ba6 data_size 0x1(0xedbe bytes) 2024-09-22T01:02:01.499 INFO:teuthology.orchestra.run.smithi002.stdout:3886 write 0xbbd0a1d thru 0xbbd0f44 (0x528 bytes) 2024-09-22T01:02:01.503 INFO:teuthology.orchestra.run.smithi002.stdout:3887 read 0x33bdef6 thru 0x33c9005 (0xb110 bytes) 2024-09-22T01:02:01.504 INFO:teuthology.orchestra.run.smithi002.stdout:3888 write 0x3d4bab0 thru 0x3d5b797 (0xfce8 bytes) 2024-09-22T01:02:01.512 INFO:teuthology.orchestra.run.smithi002.stdout:3889 punch from 0xa3880b4 to 0xa38f0f8, (0x7044 bytes) 2024-09-22T01:02:01.512 INFO:teuthology.orchestra.run.smithi002.stdout:3890 punch from 0x453f6e2 to 0x4548dd3, (0x96f1 bytes) 2024-09-22T01:02:01.512 INFO:teuthology.orchestra.run.smithi002.stdout:3891 trunc from 0xdccd275 to 0xd97a2be 2024-09-22T01:02:01.514 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:01.510+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:01.524 INFO:teuthology.orchestra.run.smithi002.stdout:3892 write 0x8d3ec4a thru 0x8d43766 (0x4b1d bytes) 2024-09-22T01:02:01.529 INFO:teuthology.orchestra.run.smithi002.stdout:3893 write 0x564ce3a thru 0x565ca2f (0xfbf6 bytes) 2024-09-22T01:02:01.538 INFO:teuthology.orchestra.run.smithi002.stdout:3894 write 0x90202c thru 0x906b54 (0x4b29 bytes) 2024-09-22T01:02:01.541 INFO:teuthology.orchestra.run.smithi002.stdout:3895 read 0x1b07245 thru 0x1b0aa61 (0x381d bytes) 2024-09-22T01:02:01.541 INFO:teuthology.orchestra.run.smithi002.stdout:3896 read 0x5b85dd6 thru 0x5b8fa21 (0x9c4c bytes) 2024-09-22T01:02:01.541 INFO:teuthology.orchestra.run.smithi002.stdout:3897 write 0x4cd5b1b thru 0x4cd8d7c (0x3262 bytes) 2024-09-22T01:02:01.545 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.545 INFO:teuthology.orchestra.run.smithi002.stdout:3899 punch from 0xbdfb494 to 0xbe0139d, (0x5f09 bytes) 2024-09-22T01:02:01.545 INFO:teuthology.orchestra.run.smithi002.stdout:3902 write 0x6925cb2 thru 0x6935749 (0xfa98 bytes) 2024-09-22T01:02:01.554 INFO:teuthology.orchestra.run.smithi002.stdout:3903 writesame 0x5986e69 thru 0x5990e41 data_size 0x1(0x9fd9 bytes) 2024-09-22T01:02:01.570 INFO:teuthology.orchestra.run.smithi002.stdout:3905 read 0x5e5dd30 thru 0x5e6a198 (0xc469 bytes) 2024-09-22T01:02:01.572 INFO:teuthology.orchestra.run.smithi002.stdout:3908 writesame 0xade86e2 thru 0xaded25b data_size 0x1(0x4b7a bytes) 2024-09-22T01:02:01.582 INFO:teuthology.orchestra.run.smithi002.stdout:3909 write 0x4481553 thru 0x4484c8c (0x373a bytes) 2024-09-22T01:02:01.587 INFO:teuthology.orchestra.run.smithi002.stdout:3911 punch from 0x33530fa to 0x33555ea, (0x24f0 bytes) 2024-09-22T01:02:01.587 INFO:teuthology.orchestra.run.smithi002.stdout:3912 read 0x12bef83 thru 0x12c57c9 (0x6847 bytes) 2024-09-22T01:02:01.587 INFO:teuthology.orchestra.run.smithi002.stdout:3913 write 0x14cc575 thru 0x14d8395 (0xbe21 bytes) 2024-09-22T01:02:01.588 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:01.589 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:01.590 INFO:teuthology.orchestra.run.smithi002.stdout:3914 read 0x258c9a2 thru 0x259218e (0x57ed bytes) 2024-09-22T01:02:01.590 INFO:teuthology.orchestra.run.smithi002.stdout:3916 punch from 0x227b257 to 0x2281271, (0x601a bytes) 2024-09-22T01:02:01.590 INFO:teuthology.orchestra.run.smithi002.stdout:3917 punch from 0xd724b49 to 0xd728b1a, (0x3fd1 bytes) 2024-09-22T01:02:01.590 INFO:teuthology.orchestra.run.smithi002.stdout:3918 read 0xc67e3ec thru 0xc6810eb (0x2d00 bytes) 2024-09-22T01:02:01.591 INFO:teuthology.orchestra.run.smithi002.stdout:3919 punch from 0x5b98962 to 0x5ba69c7, (0xe065 bytes) 2024-09-22T01:02:01.591 INFO:teuthology.orchestra.run.smithi002.stdout:3921 write 0x2cf1e0f thru 0x2cf7b68 (0x5d5a bytes) 2024-09-22T01:02:01.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:01.608 INFO:teuthology.orchestra.run.smithi002.stdout:3923 trunc from 0xd97a2be to 0x36f9290 2024-09-22T01:02:01.616 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:01.610+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:01.642 INFO:teuthology.orchestra.run.smithi002.stdout:3924 write 0x3817bd8 thru 0x381c2eb (0x4714 bytes) 2024-09-22T01:02:01.658 INFO:teuthology.orchestra.run.smithi002.stdout:3926 trunc from 0x381c2ec to 0xd907535 2024-09-22T01:02:01.666 INFO:teuthology.orchestra.run.smithi002.stdout:3928 read 0x73b4d24 thru 0x73b8aef (0x3dcc bytes) 2024-09-22T01:02:01.666 INFO:teuthology.orchestra.run.smithi002.stdout:3930 write 0x328c945 thru 0x3296557 (0x9c13 bytes) 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 78, 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-22T01:00:18.015746+0000", 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-22T01:01:59.718265+0000", 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-22T01:01:26.960327+0000", 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-22T01:01:39.066126+0000", 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5801984, 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-22T01:02:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "noscrub", 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-22T01:02:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "jewel", 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:30.699559+0000", 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-22T01:02:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-22T01:02:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:36.410605+0000", 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:02:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-22T01:02:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 2, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 21, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9900000095367432, 2024-09-22T01:02:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9900000095367432, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:46.342370+0000", 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:02:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-22T01:02:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "78", 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 33, 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 78, 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:02:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.7400000095367432, 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.7400000095367432, 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:02:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "8c0f94a8-3c63-442d-9dc1-50c1278929da", 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2277223948", 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2277223948", 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2277223948", 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2277223948", 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "2d2cf30e-e9bd-40ba-8293-193faa02ea69", 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/463340194", 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/463340194", 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/463340194", 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/463340194", 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a3d3e330-9ea1-4a5c-811b-f285f9a3cff4", 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/387643397", 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/387643397", 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/387643397", 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/387643397", 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "239072f0-4b69-4624-ba6c-733f299208da", 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 55, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-22T01:02:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-22T01:02:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-22T01:02:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-22T01:02:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1215210764", 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1215210764", 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1215210764", 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1215210764", 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fd35cb24-eb2c-4c17-abe9-ce5c34db9014", 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 0, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 54, 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6800", 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6801", 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6802", 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6803", 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6806", 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6807", 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6804", 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6805", 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6801/2463696152", 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6803/2463696152", 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6807/2463696152", 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6805/2463696152", 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:02:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "b60a584d-5334-41e4-92d9-f6689de6b8bf", 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 15, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 25, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 52, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 26, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6824", 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6825", 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6826", 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6827", 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6830", 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6831", 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6828", 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6829", 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6825/1737475131", 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6827/1737475131", 2024-09-22T01:02:01.722 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1737475131", 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1737475131", 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "9be24894-1100-4e4e-bf1b-79e163fd97e0", 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:01.723 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6816", 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6817", 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6818", 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6819", 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6822", 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6823", 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6820", 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6821", 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6817/4220217922", 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6819/4220217922", 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6823/4220217922", 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6821/4220217922", 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "293cfbc3-8c2e-40b5-ab89-0a124cc8651c", 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 67, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:01.729 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6808", 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6809", 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.730 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6810", 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6811", 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.731 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6814", 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6815", 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.732 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6812", 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6813", 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6809/3867156106", 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6811/3867156106", 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3867156106", 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3867156106", 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:02:01.734 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.556128+0000", 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:01.735 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.159753+0000", 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.766199+0000", 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.155529+0000", 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:01.737 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 65536, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.226232+0000", 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.243346+0000", 2024-09-22T01:02:01.738 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 26 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.143084+0000", 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:02:01.739 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.524369+0000", 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-22T01:02:01.740 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-22T01:02:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:01.742 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:01.743 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:01.743 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:01.744 INFO:teuthology.orchestra.run.smithi002.stdout:3932 read 0x500760b thru 0x5012398 (0xad8e bytes) 2024-09-22T01:02:01.744 INFO:teuthology.orchestra.run.smithi002.stdout:3935 write 0xa466c3f thru 0xa46e5d6 (0x7998 bytes) 2024-09-22T01:02:01.744 INFO:teuthology.orchestra.run.smithi002.stdout:3936 trunc from 0xd907535 to 0x5c7ec53 2024-09-22T01:02:01.745 INFO:tasks.thrashosds.thrasher:j is {'epoch': 78, 'fsid': '7d08b428-4185-4f0f-a0fc-f7dc1fefdce9', 'created': '2024-09-22T01:00:18.015746+0000', 'modified': '2024-09-22T01:01:59.718265+0000', 'last_up_change': '2024-09-22T01:01:26.960327+0000', 'last_in_change': '2024-09-22T01:01:39.066126+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:00:30.699559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:00:36.410605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:00:46.342370+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 78, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8c0f94a8-3c63-442d-9dc1-50c1278929da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2277223948}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2277223948}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2277223948}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2277223948}]}, 'public_addr': '172.21.15.2:6825/2277223948', 'cluster_addr': '172.21.15.2:6827/2277223948', 'heartbeat_back_addr': '172.21.15.2:6831/2277223948', 'heartbeat_front_addr': '172.21.15.2:6829/2277223948', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 463340194}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 463340194}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 463340194}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 463340194}]}, 'public_addr': '172.21.15.2:6817/463340194', 'cluster_addr': '172.21.15.2:6819/463340194', 'heartbeat_back_addr': '172.21.15.2:6823/463340194', 'heartbeat_front_addr': '172.21.15.2:6821/463340194', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 387643397}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 387643397}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 387643397}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 387643397}]}, 'public_addr': '172.21.15.2:6801/387643397', 'cluster_addr': '172.21.15.2:6803/387643397', 'heartbeat_back_addr': '172.21.15.2:6807/387643397', 'heartbeat_front_addr': '172.21.15.2:6805/387643397', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '239072f0-4b69-4624-ba6c-733f299208da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1215210764}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1215210764}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1215210764}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1215210764}]}, 'public_addr': '172.21.15.2:6809/1215210764', 'cluster_addr': '172.21.15.2:6811/1215210764', 'heartbeat_back_addr': '172.21.15.2:6815/1215210764', 'heartbeat_front_addr': '172.21.15.2:6813/1215210764', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 2463696152}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 2463696152}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 2463696152}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 2463696152}]}, 'public_addr': '172.21.15.119:6801/2463696152', 'cluster_addr': '172.21.15.119:6803/2463696152', 'heartbeat_back_addr': '172.21.15.119:6807/2463696152', 'heartbeat_front_addr': '172.21.15.119:6805/2463696152', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b60a584d-5334-41e4-92d9-f6689de6b8bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 25, 'up_from': 52, 'up_thru': 67, 'down_at': 26, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1737475131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1737475131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1737475131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1737475131}]}, 'public_addr': '172.21.15.119:6825/1737475131', 'cluster_addr': '172.21.15.119:6827/1737475131', 'heartbeat_back_addr': '172.21.15.119:6831/1737475131', 'heartbeat_front_addr': '172.21.15.119:6829/1737475131', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9be24894-1100-4e4e-bf1b-79e163fd97e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 4220217922}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 4220217922}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 4220217922}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 4220217922}]}, 'public_addr': '172.21.15.119:6817/4220217922', 'cluster_addr': '172.21.15.119:6819/4220217922', 'heartbeat_back_addr': '172.21.15.119:6823/4220217922', 'heartbeat_front_addr': '172.21.15.119:6821/4220217922', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '293cfbc3-8c2e-40b5-ab89-0a124cc8651c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3867156106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3867156106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3867156106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3867156106}]}, 'public_addr': '172.21.15.119:6809/3867156106', 'cluster_addr': '172.21.15.119:6811/3867156106', 'heartbeat_back_addr': '172.21.15.119:6815/3867156106', 'heartbeat_front_addr': '172.21.15.119:6813/3867156106', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.556128+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.159753+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.766199+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.155529+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.226232+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.243346+0000', 'dead_epoch': 26}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.143084+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.524369+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:02:01.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:02:01.746 INFO:teuthology.orchestra.run.smithi002.stdout:3937 read 0x28b7bd4 thru 0x28c6985 (0xedb2 bytes) 2024-09-22T01:02:01.747 INFO:teuthology.orchestra.run.smithi002.stdout:3939 write 0xe974cbe thru 0xe9818c3 (0xcc06 bytes) 2024-09-22T01:02:01.747 INFO:teuthology.orchestra.run.smithi002.stdout:3940 read 0x2d2574 thru 0x2d5f02 (0x398f bytes) 2024-09-22T01:02:01.747 INFO:teuthology.orchestra.run.smithi002.stdout:3941 write 0x6f6314a thru 0x6f633a3 (0x25a bytes) 2024-09-22T01:02:01.747 INFO:teuthology.orchestra.run.smithi002.stdout:3943 writesame 0xafb9da9 thru 0xafc1d4d data_size 0x1(0x7fa5 bytes) 2024-09-22T01:02:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:02:01.747 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:01.739+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:01.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:01.761 INFO:teuthology.orchestra.run.smithi002.stdout:3944 write 0x32a19b2 thru 0x32acb23 (0xb172 bytes) 2024-09-22T01:02:01.766 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.766 INFO:teuthology.orchestra.run.smithi002.stdout:3946 write 0xa24c4e2 thru 0xa25b973 (0xf492 bytes) 2024-09-22T01:02:01.773 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.773 INFO:teuthology.orchestra.run.smithi002.stdout:3948 write 0xe432643 thru 0xe43b6e4 (0x90a2 bytes) 2024-09-22T01:02:01.781 INFO:teuthology.orchestra.run.smithi002.stdout:3949 read 0xb96909e thru 0xb96d202 (0x4165 bytes) 2024-09-22T01:02:01.781 INFO:teuthology.orchestra.run.smithi002.stdout:3950 write 0xcfff2c1 thru 0xd001f3b (0x2c7b bytes) 2024-09-22T01:02:01.787 INFO:teuthology.orchestra.run.smithi002.stdout:3951 write 0x73e1927 thru 0x73e38da (0x1fb4 bytes) 2024-09-22T01:02:01.791 INFO:teuthology.orchestra.run.smithi002.stdout:3954 write 0xac75c30 thru 0xac787c0 (0x2b91 bytes) 2024-09-22T01:02:01.793 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.794 INFO:teuthology.orchestra.run.smithi002.stdout:3956 writesame 0xe7cf581 thru 0xe7dcfe7 data_size 0x1(0xda67 bytes) 2024-09-22T01:02:01.813 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.813 INFO:teuthology.orchestra.run.smithi002.stdout:3958 read 0x478329d thru 0x478946c (0x61d0 bytes) 2024-09-22T01:02:01.813 INFO:teuthology.orchestra.run.smithi002.stdout:3959 read 0x4844146 thru 0x4845499 (0x1354 bytes) 2024-09-22T01:02:01.813 INFO:teuthology.orchestra.run.smithi002.stdout:3962 write 0x15bd636 thru 0x15ccdc3 (0xf78e bytes) 2024-09-22T01:02:01.816 INFO:teuthology.orchestra.run.smithi002.stdout:3963 write 0xdc98eae thru 0xdc9c0d5 (0x3228 bytes) 2024-09-22T01:02:01.817 INFO:teuthology.orchestra.run.smithi002.stdout:3965 read 0x47d252 thru 0x48485b (0x760a bytes) 2024-09-22T01:02:01.818 INFO:teuthology.orchestra.run.smithi002.stdout:3966 write 0x6041593 thru 0x604b09c (0x9b0a bytes) 2024-09-22T01:02:01.836 INFO:teuthology.orchestra.run.smithi002.stdout:3968 trunc from 0xe9818c4 to 0x633c90a 2024-09-22T01:02:01.844 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:01.838+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:01.855 INFO:teuthology.orchestra.run.smithi002.stdout:3969 punch from 0x337d1d8 to 0x338d094, (0xfebc bytes) 2024-09-22T01:02:01.855 INFO:teuthology.orchestra.run.smithi002.stdout:3971 read 0x441b05c thru 0x44287b0 (0xd755 bytes) 2024-09-22T01:02:01.856 INFO:teuthology.orchestra.run.smithi002.stdout:3973 trunc from 0x633c90a to 0x728868d 2024-09-22T01:02:01.865 INFO:teuthology.orchestra.run.smithi002.stdout:3974 trunc from 0x728868d to 0x7d27ae0 2024-09-22T01:02:01.888 INFO:teuthology.orchestra.run.smithi002.stdout:3975 write 0xc9fb3d6 thru 0xca03836 (0x8461 bytes) 2024-09-22T01:02:01.903 INFO:teuthology.orchestra.run.smithi002.stdout:3977 write 0xceda504 thru 0xcee7e6c (0xd969 bytes) 2024-09-22T01:02:01.912 INFO:teuthology.orchestra.run.smithi002.stdout:3978 write 0x49d9479 thru 0x49dffa9 (0x6b31 bytes) 2024-09-22T01:02:01.917 INFO:teuthology.orchestra.run.smithi002.stdout:3980 write 0x6139c70 thru 0x6145187 (0xb518 bytes) 2024-09-22T01:02:01.925 INFO:teuthology.orchestra.run.smithi002.stdout:3983 punch from 0x634635c to 0x63545b8, (0xe25c bytes) 2024-09-22T01:02:01.925 INFO:teuthology.orchestra.run.smithi002.stdout:3985 punch from 0x44d8da3 to 0x44dfd6c, (0x6fc9 bytes) 2024-09-22T01:02:01.925 INFO:teuthology.orchestra.run.smithi002.stdout:3987 read 0x67949fe thru 0x6794b9a (0x19d bytes) 2024-09-22T01:02:01.925 INFO:teuthology.orchestra.run.smithi002.stdout:3989 read 0x2a2d059 thru 0x2a3aa23 (0xd9cb bytes) 2024-09-22T01:02:01.928 INFO:teuthology.orchestra.run.smithi002.stdout:3990 trunc from 0xcee7e6d to 0xac2ef2a 2024-09-22T01:02:01.929 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:01.929 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:01.944 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.945 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:01.946 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:01.943+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:01.954 INFO:teuthology.orchestra.run.smithi002.stdout:3992 write 0xc73f0e8 thru 0xc74517b (0x6094 bytes) 2024-09-22T01:02:01.963 INFO:teuthology.orchestra.run.smithi002.stdout:3993 trunc from 0xc74517c to 0xd234e08 2024-09-22T01:02:01.970 INFO:teuthology.orchestra.run.smithi002.stdout:3994 read 0x6171af thru 0x61e413 (0x7265 bytes) 2024-09-22T01:02:01.971 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:01.971 INFO:teuthology.orchestra.run.smithi002.stdout:3996 read 0xc7cec52 thru 0xc7d2de1 (0x4190 bytes) 2024-09-22T01:02:01.971 INFO:teuthology.orchestra.run.smithi002.stdout:4000 write 0xacb832c thru 0xacc4a6b (0xc740 bytes) 2024-09-22T01:02:01.980 INFO:teuthology.orchestra.run.smithi002.stdout:4002 punch from 0x153cd15 to 0x1547330, (0xa61b bytes) 2024-09-22T01:02:01.980 INFO:teuthology.orchestra.run.smithi002.stdout:4004 write 0xa575719 thru 0xa5817ce (0xc0b6 bytes) 2024-09-22T01:02:01.986 INFO:teuthology.orchestra.run.smithi002.stdout:4005 write 0x2aaf63a thru 0x2aba1dc (0xaba3 bytes) 2024-09-22T01:02:01.995 INFO:teuthology.orchestra.run.smithi002.stdout:4009 writesame 0x7485043 thru 0x7489308 data_size 0x4d(0x42c6 bytes) 2024-09-22T01:02:02.000 INFO:teuthology.orchestra.run.smithi002.stdout:4010 clone 32 order 21 su 65536 sc 5 2024-09-22T01:02:02.045 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:02.043+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:02.112 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:02:02.112 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:02.112 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:02.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:02.129 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-22T01:02:01.744670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":728827687,"num_objects":1238,"num_object_clones":365,"num_object_copies":2476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1238,"num_whiteouts":548,"num_read":115194,"num_read_kb":354672,"num_write":13712,"num_write_kb":214221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":867,"num_bytes_recovered":54550874,"num_keys_recovered":245,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11257,"ondisk_log_size":11257,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":444372,"kb_used_data":195664,"kb_used_omap":13,"kb_used_meta":248370,"kb_avail":660158508,"statfs":{"total":676457349120,"available":676002312192,"internally_reserved":0,"allocated":200359936,"data_stored":226677699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":254331124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22646788,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":11,"num_read":13498,"num_read_kb":29830,"num_write":950,"num_write_kb":6824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.527637"},"pg_stats":[{"pgid":"3.1f","version":"78'254","reported_seq":1017,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.730846+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:01:59.730846+0000","last_peered":"2024-09-22T01:01:59.730846+0000","last_clean":"2024-09-22T01:01:59.730846+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:01:59.730846+0000","last_undegraded":"2024-09-22T01:01:59.730846+0000","last_fullsized":"2024-09-22T01:01:59.730846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:57:28.286156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":27562257,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":842,"num_read_kb":9597,"num_write":286,"num_write_kb":7186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"76'560","reported_seq":2362,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.727436+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:01:59.727436+0000","last_peered":"2024-09-22T01:01:59.727436+0000","last_clean":"2024-09-22T01:01:59.727436+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:01:59.727436+0000","last_undegraded":"2024-09-22T01:01:59.727436+0000","last_fullsized":"2024-09-22T01:01:59.727436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:43:36.461411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":25375920,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":4809,"num_read_kb":9919,"num_write":814,"num_write_kb":6085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"78'293","reported_seq":3446,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.724835+0000","last_change":"2024-09-22T01:01:43.138890+0000","last_active":"2024-09-22T01:02:00.724835+0000","last_peered":"2024-09-22T01:02:00.724835+0000","last_clean":"2024-09-22T01:02:00.724835+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:02:00.724835+0000","last_undegraded":"2024-09-22T01:02:00.724835+0000","last_fullsized":"2024-09-22T01:02:00.724835+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:32:28.886278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32573436,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":3893,"num_read_kb":17097,"num_write":292,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"78'831","reported_seq":3529,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.724754+0000","last_change":"2024-09-22T01:01:43.141896+0000","last_active":"2024-09-22T01:02:00.724754+0000","last_peered":"2024-09-22T01:02:00.724754+0000","last_clean":"2024-09-22T01:02:00.724754+0000","last_became_active":"2024-09-22T01:01:43.141151+0000","last_became_peered":"2024-09-22T01:01:43.141151+0000","last_unstale":"2024-09-22T01:02:00.724754+0000","last_undegraded":"2024-09-22T01:02:00.724754+0000","last_fullsized":"2024-09-22T01:02:00.724754+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:23.089426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49439782,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":5430,"num_read_kb":29819,"num_write":1261,"num_write_kb":10888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":9399679,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'328","reported_seq":688,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.730846+0000","last_change":"2024-09-22T01:01:43.147132+0000","last_active":"2024-09-22T01:01:59.730846+0000","last_peered":"2024-09-22T01:01:59.730846+0000","last_clean":"2024-09-22T01:01:59.730846+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:01:59.730846+0000","last_undegraded":"2024-09-22T01:01:59.730846+0000","last_fullsized":"2024-09-22T01:01:59.730846+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:07.553460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":16309577,"num_objects":39,"num_object_clones":16,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":904,"num_read_kb":1054,"num_write":402,"num_write_kb":1022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"78'252","reported_seq":2318,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.604052+0000","last_change":"2024-09-22T01:01:43.143586+0000","last_active":"2024-09-22T01:02:00.604052+0000","last_peered":"2024-09-22T01:02:00.604052+0000","last_clean":"2024-09-22T01:02:00.604052+0000","last_became_active":"2024-09-22T01:01:43.138644+0000","last_became_peered":"2024-09-22T01:01:43.138644+0000","last_unstale":"2024-09-22T01:02:00.604052+0000","last_undegraded":"2024-09-22T01:02:00.604052+0000","last_fullsized":"2024-09-22T01:02:00.604052+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:05:12.484492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":26709770,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":2237,"num_read_kb":4586,"num_write":334,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":156984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"77'201","reported_seq":3636,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.089074+0000","last_change":"2024-09-22T01:00:49.708786+0000","last_active":"2024-09-22T01:02:00.089074+0000","last_peered":"2024-09-22T01:02:00.089074+0000","last_clean":"2024-09-22T01:02:00.089074+0000","last_became_active":"2024-09-22T01:00:47.723339+0000","last_became_peered":"2024-09-22T01:00:47.723339+0000","last_unstale":"2024-09-22T01:02:00.089074+0000","last_undegraded":"2024-09-22T01:02:00.089074+0000","last_fullsized":"2024-09-22T01:02:00.089074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:40:45.577757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":25429920,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":11,"num_read":3388,"num_read_kb":14758,"num_write":229,"num_write_kb":6090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"78'496","reported_seq":4894,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.639333+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:02:00.639333+0000","last_peered":"2024-09-22T01:02:00.639333+0000","last_clean":"2024-09-22T01:02:00.639333+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:02:00.639333+0000","last_undegraded":"2024-09-22T01:02:00.639333+0000","last_fullsized":"2024-09-22T01:02:00.639333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:01.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":44746609,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":6809,"num_read_kb":27972,"num_write":614,"num_write_kb":7079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"78'359","reported_seq":2969,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.589389+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:02:00.589389+0000","last_peered":"2024-09-22T01:02:00.589389+0000","last_clean":"2024-09-22T01:02:00.589389+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:02:00.589389+0000","last_undegraded":"2024-09-22T01:02:00.589389+0000","last_fullsized":"2024-09-22T01:02:00.589389+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:09:28.996458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":14422509,"num_objects":42,"num_object_clones":15,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":20,"num_read":3632,"num_read_kb":9939,"num_write":444,"num_write_kb":4120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"78'207","reported_seq":752,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.948292+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:02:00.948292+0000","last_peered":"2024-09-22T01:02:00.948292+0000","last_clean":"2024-09-22T01:02:00.948292+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:02:00.948292+0000","last_undegraded":"2024-09-22T01:02:00.948292+0000","last_fullsized":"2024-09-22T01:02:00.948292+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:29:59.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19870404,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":1656,"num_read_kb":9557,"num_write":201,"num_write_kb":13520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"73'416","reported_seq":5106,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.087347+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:02:00.087347+0000","last_peered":"2024-09-22T01:02:00.087347+0000","last_clean":"2024-09-22T01:02:00.087347+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:02:00.087347+0000","last_undegraded":"2024-09-22T01:02:00.087347+0000","last_fullsized":"2024-09-22T01:02:00.087347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:11.591064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":22012870,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":6673,"num_read_kb":19248,"num_write":451,"num_write_kb":1531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"78'352","reported_seq":8569,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.593696+0000","last_change":"2024-09-22T01:01:43.139393+0000","last_active":"2024-09-22T01:02:00.593696+0000","last_peered":"2024-09-22T01:02:00.593696+0000","last_clean":"2024-09-22T01:02:00.593696+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:02:00.593696+0000","last_undegraded":"2024-09-22T01:02:00.593696+0000","last_fullsized":"2024-09-22T01:02:00.593696+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:04.953232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":31735944,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":8978,"num_read_kb":18841,"num_write":390,"num_write_kb":8383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.804157+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:01:59.804157+0000","last_peered":"2024-09-22T01:01:59.804157+0000","last_clean":"2024-09-22T01:01:59.804157+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:01:59.804157+0000","last_undegraded":"2024-09-22T01:01:59.804157+0000","last_fullsized":"2024-09-22T01:01:59.804157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"78'274","reported_seq":1604,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.730422+0000","last_change":"2024-09-22T01:00:49.729709+0000","last_active":"2024-09-22T01:01:59.730422+0000","last_peered":"2024-09-22T01:01:59.730422+0000","last_clean":"2024-09-22T01:01:59.730422+0000","last_became_active":"2024-09-22T01:00:47.718964+0000","last_became_peered":"2024-09-22T01:00:47.718964+0000","last_unstale":"2024-09-22T01:01:59.730422+0000","last_undegraded":"2024-09-22T01:01:59.730422+0000","last_fullsized":"2024-09-22T01:01:59.730422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:49.935970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":15923247,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":2948,"num_read_kb":4292,"num_write":254,"num_write_kb":1302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.724436+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:01:59.724436+0000","last_peered":"2024-09-22T01:01:59.724436+0000","last_clean":"2024-09-22T01:01:59.724436+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:01:59.724436+0000","last_undegraded":"2024-09-22T01:01:59.724436+0000","last_fullsized":"2024-09-22T01:01:59.724436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"78'378","reported_seq":1362,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.524702+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:02:00.524702+0000","last_peered":"2024-09-22T01:02:00.524702+0000","last_clean":"2024-09-22T01:02:00.524702+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:02:00.524702+0000","last_undegraded":"2024-09-22T01:02:00.524702+0000","last_fullsized":"2024-09-22T01:02:00.524702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:09:49.848340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":13547456,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":2024,"num_read_kb":6370,"num_write":531,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.779955+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:01:59.779955+0000","last_peered":"2024-09-22T01:01:59.779955+0000","last_clean":"2024-09-22T01:01:59.779955+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:01:59.779955+0000","last_undegraded":"2024-09-22T01:01:59.779955+0000","last_fullsized":"2024-09-22T01:01:59.779955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"77'240","reported_seq":2869,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.724752+0000","last_change":"2024-09-22T01:00:49.709328+0000","last_active":"2024-09-22T01:01:59.724752+0000","last_peered":"2024-09-22T01:01:59.724752+0000","last_clean":"2024-09-22T01:01:59.724752+0000","last_became_active":"2024-09-22T01:00:47.714621+0000","last_became_peered":"2024-09-22T01:00:47.714621+0000","last_unstale":"2024-09-22T01:01:59.724752+0000","last_undegraded":"2024-09-22T01:01:59.724752+0000","last_fullsized":"2024-09-22T01:01:59.724752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:16.209914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":10929156,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":3131,"num_read_kb":11854,"num_write":208,"num_write_kb":6356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.724743+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:01:59.724743+0000","last_peered":"2024-09-22T01:01:59.724743+0000","last_clean":"2024-09-22T01:01:59.724743+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:01:59.724743+0000","last_undegraded":"2024-09-22T01:01:59.724743+0000","last_fullsized":"2024-09-22T01:01:59.724743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"78'355","reported_seq":2717,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.948592+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:02:00.948592+0000","last_peered":"2024-09-22T01:02:00.948592+0000","last_clean":"2024-09-22T01:02:00.948592+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:02:00.948592+0000","last_undegraded":"2024-09-22T01:02:00.948592+0000","last_fullsized":"2024-09-22T01:02:00.948592+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:35:18.517475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18027888,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":4810,"num_read_kb":8118,"num_write":327,"num_write_kb":2738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"78'244","reported_seq":856,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.517286+0000","last_change":"2024-09-22T01:01:43.148315+0000","last_active":"2024-09-22T01:02:00.517286+0000","last_peered":"2024-09-22T01:02:00.517286+0000","last_clean":"2024-09-22T01:02:00.517286+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:02:00.517286+0000","last_undegraded":"2024-09-22T01:02:00.517286+0000","last_fullsized":"2024-09-22T01:02:00.517286+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:54:22.721626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":19137669,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":849,"num_read_kb":6161,"num_write":282,"num_write_kb":6743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":88,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.736755+0000","last_change":"2024-09-22T01:01:27.722404+0000","last_active":"2024-09-22T01:01:59.736755+0000","last_peered":"2024-09-22T01:01:59.736755+0000","last_clean":"2024-09-22T01:01:59.736755+0000","last_became_active":"2024-09-22T01:01:27.721384+0000","last_became_peered":"2024-09-22T01:01:27.721384+0000","last_unstale":"2024-09-22T01:01:59.736755+0000","last_undegraded":"2024-09-22T01:01:59.736755+0000","last_fullsized":"2024-09-22T01:01:59.736755+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:10.574832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"78'1235","reported_seq":2264,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.649572+0000","last_change":"2024-09-22T01:00:49.709391+0000","last_active":"2024-09-22T01:02:00.649572+0000","last_peered":"2024-09-22T01:02:00.649572+0000","last_clean":"2024-09-22T01:02:00.649572+0000","last_became_active":"2024-09-22T01:00:47.714544+0000","last_became_peered":"2024-09-22T01:00:47.714544+0000","last_unstale":"2024-09-22T01:02:00.649572+0000","last_undegraded":"2024-09-22T01:02:00.649572+0000","last_fullsized":"2024-09-22T01:02:00.649572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:45:40.218411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":23393103,"num_objects":42,"num_object_clones":14,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":18,"num_read":7146,"num_read_kb":14755,"num_write":2137,"num_write_kb":7575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.722965+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:01:59.722965+0000","last_peered":"2024-09-22T01:01:59.722965+0000","last_clean":"2024-09-22T01:01:59.722965+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:01:59.722965+0000","last_undegraded":"2024-09-22T01:01:59.722965+0000","last_fullsized":"2024-09-22T01:01:59.722965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"76'280","reported_seq":1721,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.730390+0000","last_change":"2024-09-22T01:01:43.152419+0000","last_active":"2024-09-22T01:01:59.730390+0000","last_peered":"2024-09-22T01:01:59.730390+0000","last_clean":"2024-09-22T01:01:59.730390+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:01:59.730390+0000","last_undegraded":"2024-09-22T01:01:59.730390+0000","last_fullsized":"2024-09-22T01:01:59.730390+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:32.062158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":9897047,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":1544,"num_read_kb":6510,"num_write":303,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.778986+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:01:59.778986+0000","last_peered":"2024-09-22T01:01:59.778986+0000","last_clean":"2024-09-22T01:01:59.778986+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:01:59.778986+0000","last_undegraded":"2024-09-22T01:01:59.778986+0000","last_fullsized":"2024-09-22T01:01:59.778986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"78'251","reported_seq":1475,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.727515+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:01:59.727515+0000","last_peered":"2024-09-22T01:01:59.727515+0000","last_clean":"2024-09-22T01:01:59.727515+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:01:59.727515+0000","last_undegraded":"2024-09-22T01:01:59.727515+0000","last_fullsized":"2024-09-22T01:01:59.727515+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:58:18.828501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":15032775,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":1318,"num_read_kb":5736,"num_write":272,"num_write_kb":11861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":87,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.736835+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:01:59.736835+0000","last_peered":"2024-09-22T01:01:59.736835+0000","last_clean":"2024-09-22T01:01:59.736835+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:01:59.736835+0000","last_undegraded":"2024-09-22T01:01:59.736835+0000","last_fullsized":"2024-09-22T01:01:59.736835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":284,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.779008+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:01:59.779008+0000","last_peered":"2024-09-22T01:01:59.779008+0000","last_clean":"2024-09-22T01:01:59.779008+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:01:59.779008+0000","last_undegraded":"2024-09-22T01:01:59.779008+0000","last_fullsized":"2024-09-22T01:01:59.779008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"78'276","reported_seq":8154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.182774+0000","last_change":"2024-09-22T01:00:49.735787+0000","last_active":"2024-09-22T01:02:00.182774+0000","last_peered":"2024-09-22T01:02:00.182774+0000","last_clean":"2024-09-22T01:02:00.182774+0000","last_became_active":"2024-09-22T01:00:47.720325+0000","last_became_peered":"2024-09-22T01:00:47.720325+0000","last_unstale":"2024-09-22T01:02:00.182774+0000","last_undegraded":"2024-09-22T01:02:00.182774+0000","last_fullsized":"2024-09-22T01:02:00.182774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:52.246615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":23345439,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":9076,"num_read_kb":14485,"num_write":235,"num_write_kb":3870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"78'295","reported_seq":917,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.519086+0000","last_change":"2024-09-22T01:01:43.142999+0000","last_active":"2024-09-22T01:02:00.519086+0000","last_peered":"2024-09-22T01:02:00.519086+0000","last_clean":"2024-09-22T01:02:00.519086+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:02:00.519086+0000","last_undegraded":"2024-09-22T01:02:00.519086+0000","last_fullsized":"2024-09-22T01:02:00.519086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:09:47.917392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":22843457,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":1026,"num_read_kb":2975,"num_write":350,"num_write_kb":5139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"78'446","reported_seq":9782,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:01.615326+0000","last_change":"2024-09-22T01:01:43.139528+0000","last_active":"2024-09-22T01:02:01.615326+0000","last_peered":"2024-09-22T01:02:01.615326+0000","last_clean":"2024-09-22T01:02:01.615326+0000","last_became_active":"2024-09-22T01:01:43.138786+0000","last_became_peered":"2024-09-22T01:01:43.138786+0000","last_unstale":"2024-09-22T01:02:01.615326+0000","last_undegraded":"2024-09-22T01:02:01.615326+0000","last_fullsized":"2024-09-22T01:02:01.615326+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:34:22.445137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":20647426,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":17,"num_read":10362,"num_read_kb":12458,"num_write":491,"num_write_kb":5831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":167229,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"77'356","reported_seq":1622,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.086199+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:02:00.086199+0000","last_peered":"2024-09-22T01:02:00.086199+0000","last_clean":"2024-09-22T01:02:00.086199+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:02:00.086199+0000","last_undegraded":"2024-09-22T01:02:00.086199+0000","last_fullsized":"2024-09-22T01:02:00.086199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:15:57.767043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":8329915,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":2959,"num_read_kb":14496,"num_write":427,"num_write_kb":6750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"78'278","reported_seq":558,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.939447+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:02:00.939447+0000","last_peered":"2024-09-22T01:02:00.939447+0000","last_clean":"2024-09-22T01:02:00.939447+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:02:00.939447+0000","last_undegraded":"2024-09-22T01:02:00.939447+0000","last_fullsized":"2024-09-22T01:02:00.939447+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:22:49.832894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21660616,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":2231,"num_read_kb":11199,"num_write":281,"num_write_kb":4273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"78'285","reported_seq":3832,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.057522+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:02:00.057522+0000","last_peered":"2024-09-22T01:02:00.057522+0000","last_clean":"2024-09-22T01:02:00.057522+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:02:00.057522+0000","last_undegraded":"2024-09-22T01:02:00.057522+0000","last_fullsized":"2024-09-22T01:02:00.057522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:11.226937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":25247857,"num_objects":47,"num_object_clones":18,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":4579,"num_read_kb":22059,"num_write":268,"num_write_kb":13013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"73'238","reported_seq":2397,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.087163+0000","last_change":"2024-09-22T01:01:43.140327+0000","last_active":"2024-09-22T01:02:00.087163+0000","last_peered":"2024-09-22T01:02:00.087163+0000","last_clean":"2024-09-22T01:02:00.087163+0000","last_became_active":"2024-09-22T01:01:43.139857+0000","last_became_peered":"2024-09-22T01:01:43.139857+0000","last_unstale":"2024-09-22T01:02:00.087163+0000","last_undegraded":"2024-09-22T01:02:00.087163+0000","last_fullsized":"2024-09-22T01:02:00.087163+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:18:05.153665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":37965511,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":1910,"num_read_kb":11168,"num_write":242,"num_write_kb":11530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":2830133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"78'200","reported_seq":3496,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.570714+0000","last_change":"2024-09-22T01:01:43.142453+0000","last_active":"2024-09-22T01:02:00.570714+0000","last_peered":"2024-09-22T01:02:00.570714+0000","last_clean":"2024-09-22T01:02:00.570714+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:02:00.570714+0000","last_undegraded":"2024-09-22T01:02:00.570714+0000","last_fullsized":"2024-09-22T01:02:00.570714+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:30:28.686550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":14084877,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":3844,"num_read_kb":4139,"num_write":188,"num_write_kb":7389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"78'270","reported_seq":2304,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:01.615755+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:02:01.615755+0000","last_peered":"2024-09-22T01:02:01.615755+0000","last_clean":"2024-09-22T01:02:01.615755+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:02:01.615755+0000","last_undegraded":"2024-09-22T01:02:01.615755+0000","last_fullsized":"2024-09-22T01:02:01.615755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:21:03.548915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":36638823,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":2532,"num_read_kb":9123,"num_write":225,"num_write_kb":7640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"78'316","reported_seq":1165,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.628317+0000","last_change":"2024-09-22T01:01:43.139843+0000","last_active":"2024-09-22T01:02:00.628317+0000","last_peered":"2024-09-22T01:02:00.628317+0000","last_clean":"2024-09-22T01:02:00.628317+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:02:00.628317+0000","last_undegraded":"2024-09-22T01:02:00.628317+0000","last_fullsized":"2024-09-22T01:02:00.628317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:25:58.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":18242572,"num_objects":36,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":1658,"num_read_kb":3722,"num_write":389,"num_write_kb":1889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"78'195","reported_seq":1253,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:02:00.662034+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:02:00.662034+0000","last_peered":"2024-09-22T01:02:00.662034+0000","last_clean":"2024-09-22T01:02:00.662034+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:02:00.662034+0000","last_undegraded":"2024-09-22T01:02:00.662034+0000","last_fullsized":"2024-09-22T01:02:00.662034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:55.255779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":22447960,"num_objects":42,"num_object_clones":15,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":20,"num_read":1038,"num_read_kb":2607,"num_write":173,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"73'176","reported_seq":1154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T01:01:59.779575+0000","last_change":"2024-09-22T01:01:43.141814+0000","last_active":"2024-09-22T01:01:59.779575+0000","last_peered":"2024-09-22T01:01:59.779575+0000","last_clean":"2024-09-22T01:01:59.779575+0000","last_became_active":"2024-09-22T01:01:43.141276+0000","last_became_peered":"2024-09-22T01:01:43.141276+0000","last_unstale":"2024-09-22T01:01:59.779575+0000","last_undegraded":"2024-09-22T01:01:59.779575+0000","last_fullsized":"2024-09-22T01:01:59.779575+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:30.051245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":14705508,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":842,"num_read_kb":9958,"num_write":250,"num_write_kb":9590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":5873425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":728237300,"num_objects":1234,"num_object_clones":365,"num_object_copies":2468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1234,"num_whiteouts":548,"num_read":115078,"num_read_kb":354572,"num_write":13551,"num_write_kb":211803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":867,"num_bytes_recovered":54550874,"num_keys_recovered":245,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":220803072,"data_stored":254657665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":11137,"ondisk_log_size":11137,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59412,"kb_used_data":24060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94312428,"statfs":{"total":96636764160,"available":96575926272,"internally_reserved":0,"allocated":24637440,"data_stored":27696205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299400,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75044,"kb_used_data":38608,"kb_used_omap":4,"kb_used_meta":36411,"kb_avail":94296796,"statfs":{"total":96636764160,"available":96559919104,"internally_reserved":0,"allocated":39534592,"data_stored":47543405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":37285454},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70676,"kb_used_data":31164,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94301164,"statfs":{"total":96636764160,"available":96564391936,"internally_reserved":0,"allocated":31911936,"data_stored":37130797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542166,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63204,"kb_used_data":32016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308636,"statfs":{"total":96636764160,"available":96572043264,"internally_reserved":0,"allocated":32784384,"data_stored":35015857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58820,"kb_used_data":14976,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94313020,"statfs":{"total":96636764160,"available":96576532480,"internally_reserved":0,"allocated":15335424,"data_stored":17118604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54304,"kb_used_data":23116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317536,"statfs":{"total":96636764160,"available":96581156864,"internally_reserved":0,"allocated":23670784,"data_stored":27896569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62912,"kb_used_data":31724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308928,"statfs":{"total":96636764160,"available":96572342272,"internally_reserved":0,"allocated":32485376,"data_stored":34276262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31035392,"data_stored":33309660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22777856,"data_stored":27495968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14426112,"data_stored":16701525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31973376,"data_stored":34686575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":39559168,"data_stored":48042497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23760896,"data_stored":27303772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":30478336,"data_stored":36181671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:02:02.131 INFO:tasks.thrashosds.thrasher:Setting 3.13 to [1, 5, 7, 3] 2024-09-22T01:02:02.131 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.13', '1', '5', '7', '3'] 2024-09-22T01:02:02.132 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.13 1 5 7 3 2024-09-22T01:02:02.174 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:02.174+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:02.247 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:02.243+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:02.291 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:02.291 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:02.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:02.348 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:02.347+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:02.448 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:02.442+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:02.471 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:02.471 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:02.485 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:02.548 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:02.542+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:02.613 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:02.613 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:02.628 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:02.649 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:02.647+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:02.749 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:02.747+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:set 3.13 pg_upmap_items mapping to [1->5,7->3] 2024-09-22T01:02:02.809 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:02.809 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:02.826 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:02.850 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:02.847+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:02.950 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:02.947+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:02.993 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:02.993 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:03.008 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:03.010 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-22T01:02:03.051 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:03.047+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:03.144 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:03.144 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:03.152 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:03.146+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:03.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:03.252 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:03.250+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:03.342 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:03.386 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:03.386+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:03.417 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:03.454 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:03.451+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:03.506 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:03.506 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:03.522 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:03.569 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:03.566+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:03.580 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:03.580 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:03.597 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:03.597 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:03.615 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:03.615 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:03.632 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:03.633 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:03.646 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:03.646 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:03.646 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:03.646 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:03.653 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:03.653 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:03.654 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:03.650+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:03.660 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:03.676 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:03.676 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:03.698 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:03.698 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:03.722 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:03.722 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:03.775 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:02:03.786 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:03.786+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:03.801 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:03.801 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:03.801 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:03.801 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:03.801 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:03.816 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:03.858 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:03.854+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:03.957 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:03.955+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8964 3.4 e32/22 3:20bcb6f7:::rbd_data.117cc87baa96.0000000000000002:head v 32'20, mlcod=32'17)", 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.839340527000004, 2024-09-22T01:02:03.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077894110000000004, 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136523+0000", 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.136525+0000", 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3540000000000001e-06 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137650+0000", 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011257190000000001 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137671+0000", 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0560999999999998e-05 2024-09-22T01:02:03.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137680+0000", 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137763+0000", 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2472999999999995e-05 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:03.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.137793+0000", 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-05 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144267+0000", 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064739139999999999 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.144313+0000", 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5624999999999998e-05 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:03.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:8968 3.4 e32/22 3:21e3db6e:::rbd_data.117cc87baa96.0000000000000003:head v 32'21, mlcod=32'17)", 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.834914482000002, 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004894771, 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140949+0000", 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.140952+0000", 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1849999999999998e-06 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:03.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.141985+0000", 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001032984 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142001+0000", 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6776999999999999e-05 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142010+0000", 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4640000000000006e-06 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142101+0000", 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1312999999999998e-05 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.142119+0000", 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8202e-05 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145810+0000", 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036909970000000001 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:59.145844+0000", 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3849000000000001e-05 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:35894 3.0 e48/22 3:07da6ff3:::rbd_data.117cd9a1164e.0000000000000003:head v 48'145, mlcod=47'143)", 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.822849230000003, 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055540160000000002, 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:03.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153015+0000", 2024-09-22T01:02:03.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8700000000000002e-07 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153852+0000", 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083735000000000005 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153865+0000", 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2804e-05 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:03.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153872+0000", 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153943+0000", 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0908000000000006e-05 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.153968+0000", 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5530000000000001e-05 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158535+0000", 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045670559999999999 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:22.158569+0000", 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210999999999998e-05 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:37899 3.9 e49/22 3:91d7590e:::rbd_data.117cf8aab9ed.0000000000000003:head v 49'73, mlcod=48'71)", 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:02:03.992 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 40.796345406999997, 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012095786000000001, 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179518+0000", 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.179519+0000", 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4099999999999997e-07 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183241+0000", 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037216110000000001 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183261+0000", 2024-09-22T01:02:03.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9799000000000001e-05 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183274+0000", 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3046e-05 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183374+0000", 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010009399999999999 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.183399+0000", 2024-09-22T01:02:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5375999999999999e-05 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191540+0000", 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081403029999999998 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:23.191614+0000", 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4715999999999994e-05 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:43527 3.9 e60/22 3:96fe8936:::rbd_data.117c83c8bd76.000000000000001b:head v 60'182, mlcod=60'180)", 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.134294195999999, 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005051725, 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841570+0000", 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.841572+0000", 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845187+0000", 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036148880000000001 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845210+0000", 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2926e-05 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845221+0000", 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.17e-05 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845322+0000", 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101003 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:03.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.845348+0000", 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5361e-05 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846545+0000", 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011971659999999999 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:35.846621+0000", 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6631000000000005e-05 2024-09-22T01:02:04.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56176 3.9 3:91eeadac:::rbd_data.117c93605fe7.00000000000001f7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.077109892999999, 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038039409999999999, 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56176 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898754+0000", 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898756+0000", 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.832e-06 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898763+0000", 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3440000000000003e-06 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898778+0000", 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4982e-05 2024-09-22T01:02:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898787+0000", 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7479999999999999e-06 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898845+0000", 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8616999999999998e-05 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898903+0000", 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7623e-05 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.898982+0000", 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9124999999999999e-05 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.899559+0000", 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00057709899999999997 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902402+0000", 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028426990000000002 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902461+0000", 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8607999999999999e-05 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902558+0000", 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7263999999999996e-05 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56184 3.9 3:914e0854:::rbd_data.117c93605fe7.000000000000021b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.07450145, 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040309569999999999, 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56184 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901362+0000", 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901364+0000", 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.48e-06 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901369+0000", 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1050000000000003e-06 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901384+0000", 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5098e-05 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901392+0000", 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9969999999999998e-06 2024-09-22T01:02:04.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901444+0000", 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2268999999999997e-05 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901496+0000", 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1864999999999999e-05 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.901562+0000", 2024-09-22T01:02:04.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5718999999999999e-05 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.902103+0000", 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054061300000000001 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905243+0000", 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031400059999999999 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905305+0000", 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2527000000000006e-05 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.905393+0000", 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8277999999999997e-05 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:56402 3.9 3:9338a8f3:::rbd_data.117c93605fe7.00000000000002a2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.900653132999999, 2024-09-22T01:02:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004133734, 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56402 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075211+0000", 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075212+0000", 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7060000000000001e-06 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075220+0000", 2024-09-22T01:02:04.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2590000000000003e-06 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075235+0000", 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.537e-05 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075243+0000", 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1159999999999999e-06 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075314+0000", 2024-09-22T01:02:04.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0535000000000001e-05 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075399+0000", 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5745e-05 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.075512+0000", 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112575 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.076225+0000", 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071304799999999998 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079253+0000", 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030281520000000001 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079290+0000", 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7240000000000003e-05 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:42.079344+0000", 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3987999999999999e-05 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:58441 3.9 3:961db89f:::rbd_data.117c93605fe7.000000000000027c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.233992999000002, 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035419259999999999, 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 58441 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741871+0000", 2024-09-22T01:02:04.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741873+0000", 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7990000000000001e-06 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741879+0000", 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0249999999999999e-06 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741896+0000", 2024-09-22T01:02:04.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6962000000000001e-05 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741904+0000", 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4489999999999999e-06 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.741961+0000", 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7206000000000001e-05 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742048+0000", 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7078999999999998e-05 2024-09-22T01:02:04.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742156+0000", 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010733799999999999 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.742877+0000", 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072088000000000002 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745306+0000", 2024-09-22T01:02:04.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002429512 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745365+0000", 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8962000000000002e-05 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:44.745413+0000", 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7713999999999998e-05 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59024 3.9 3:935ab33d:::rbd_data.117ce35b6780.00000000000001db:head [set-alloc-hint object_size 262144 write_size 262144,write 33792~3072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.258278198999999, 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048336580000000002, 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59024 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717586+0000", 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717588+0000", 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8030000000000001e-06 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717600+0000", 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2028000000000001e-05 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717626+0000", 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5930000000000001e-05 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717639+0000", 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2296000000000001e-05 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717728+0000", 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9091999999999993e-05 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.717864+0000", 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013604200000000001 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.718008+0000", 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014423800000000001 2024-09-22T01:02:04.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.719029+0000", 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001020897 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722322+0000", 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032925509999999999 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722370+0000", 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8755000000000003e-05 2024-09-22T01:02:04.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.722419+0000", 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9026000000000001e-05 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59026 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [set-alloc-hint object_size 262144 write_size 262144,write 32768~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.255609104000001, 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046681709999999996, 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59026 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720255+0000", 2024-09-22T01:02:04.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720256+0000", 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3969999999999999e-06 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720267+0000", 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0699e-05 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720285+0000", 2024-09-22T01:02:04.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8561999999999999e-05 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720296+0000", 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0324e-05 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720362+0000", 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6156999999999997e-05 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720474+0000", 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111656 2024-09-22T01:02:04.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.720641+0000", 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167875 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.721449+0000", 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00080787999999999997 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724779+0000", 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033294090000000002 2024-09-22T01:02:04.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724864+0000", 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5535e-05 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.724923+0000", 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8677000000000001e-05 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59035 3.9 3:96fe214a:::rbd_data.117ce35b6780.00000000000001dd:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.236671134000002, 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056261260000000004, 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59035 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739193+0000", 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739194+0000", 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7599999999999996e-07 2024-09-22T01:02:04.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739199+0000", 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1919999999999996e-06 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739212+0000", 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3484e-05 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739222+0000", 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3279999999999995e-06 2024-09-22T01:02:04.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739864+0000", 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064251300000000005 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.739973+0000", 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010924700000000001 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740123+0000", 2024-09-22T01:02:04.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014958199999999999 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.740800+0000", 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067721600000000002 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744735+0000", 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039345889999999996 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744779+0000", 2024-09-22T01:02:04.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4196999999999997e-05 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.744819+0000", 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0002000000000001e-05 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:59186 3.9 3:973f8b5e:::rbd_data.117ce35b6780.000000000000007d:head [set-alloc-hint object_size 262144 write_size 262144,write 131217~7023] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.977808102000001, 2024-09-22T01:02:04.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039873030000000002, 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59186 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998056+0000", 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998059+0000", 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998076+0000", 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.732e-05 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998110+0000", 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3785999999999998e-05 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998124+0000", 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4544e-05 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998226+0000", 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101544 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998436+0000", 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021000899999999999 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.998697+0000", 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000261406 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:45.999737+0000", 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010393519999999999 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001927+0000", 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021903420000000001 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.001985+0000", 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7711999999999999e-05 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:46.002043+0000", 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8477999999999998e-05 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:63727 3.9 3:95e8888c:::rbd_data.117c12a7c618.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.903841718000001, 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057114310000000003, 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 63727 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072022+0000", 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072037+0000", 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4440999999999999e-05 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072044+0000", 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1539999999999996e-06 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072059+0000", 2024-09-22T01:02:04.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4936000000000001e-05 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072066+0000", 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.464e-06 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072130+0000", 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4054999999999996e-05 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.072249+0000", 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011866099999999999 2024-09-22T01:02:04.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.073379+0000", 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011302040000000001 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077601+0000", 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042215810000000003 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077622+0000", 2024-09-22T01:02:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1622999999999999e-05 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077686+0000", 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4003000000000004e-05 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:51.077734+0000", 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7308999999999997e-05 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:77746 3.9 3:9322efa7:::rbd_data.117c2270b223.0000000000000016:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2529512~54077] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:55.119750+0000", 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.8561135330000003, 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037591759999999999, 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 77746 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119750+0000", 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119750+0000", 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119752+0000", 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.815e-06 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119800+0000", 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7574999999999998e-05 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119822+0000", 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2167e-05 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119834+0000", 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2269000000000001e-05 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.119899+0000", 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5110000000000005e-05 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.120046+0000", 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014652 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.120205+0000", 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158949 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.121429+0000", 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012244879999999999 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.123342+0000", 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019131969999999999 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.123439+0000", 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6495000000000004e-05 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:55.123510+0000", 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0591000000000002e-05 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:84473 3.0 e78/22 3:01606850:::rbd_object_map.117cbccc292e:head v 78'1204, mlcod=78'1202)", 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:00.488812+0000", 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4870518530000001, 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049345470000000001, 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.057 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.488812+0000", 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.488812+0000", 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.488814+0000", 2024-09-22T01:02:04.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.488831+0000", 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7368999999999999e-05 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.488872+0000", 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0708999999999999e-05 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.488893+0000", 2024-09-22T01:02:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0466e-05 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.489047+0000", 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015458099999999999 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.489110+0000", 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2742000000000003e-05 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.493639+0000", 2024-09-22T01:02:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045286529999999997 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.493747+0000", 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107957 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:84506 3.0 e78/22 3:01606850:::rbd_object_map.117cbccc292e:head v 78'1209, mlcod=78'1202)", 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:00.506269+0000", 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.469594555, 2024-09-22T01:02:04.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039030319999999999, 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.506269+0000", 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.506269+0000", 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.506271+0000", 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7430000000000001e-06 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.506286+0000", 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4993e-05 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.506309+0000", 2024-09-22T01:02:04.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3413000000000001e-05 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.506328+0000", 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8850000000000001e-05 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.506493+0000", 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016461000000000001 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.506527+0000", 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4338999999999998e-05 2024-09-22T01:02:04.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.510110+0000", 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035829189999999999 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.510172+0000", 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2164999999999999e-05 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:84518 3.9 3:97e783bb:::rbd_data.117cbccc292e.0000000000000075:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,zero 1228800~145238] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:00.515136+0000", 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4607277989999998, 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040391419999999999, 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 84518 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.515136+0000", 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.515136+0000", 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.515139+0000", 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.428e-06 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.515158+0000", 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8916999999999999e-05 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.515194+0000", 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5542e-05 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.515213+0000", 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516117+0000", 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090421999999999996 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516349+0000", 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023141599999999999 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516719+0000", 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037065399999999999 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.518342+0000", 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016221639999999999 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.518956+0000", 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061396699999999998 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.519083+0000", 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012785199999999999 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.519175+0000", 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1724999999999993e-05 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:84510 3.0 e78/22 3:01606850:::rbd_object_map.117cbccc292e:head v 78'1212, mlcod=78'1210)", 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:00.516254+0000", 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4596094110000002, 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036875050000000002, 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516254+0000", 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516254+0000", 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516256+0000", 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.449e-06 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516267+0000", 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0878e-05 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516285+0000", 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7778000000000002e-05 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516297+0000", 2024-09-22T01:02:04.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2306e-05 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516464+0000", 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016761599999999999 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.516496+0000", 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1398e-05 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.519880+0000", 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033839809999999999 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.519942+0000", 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2099000000000003e-05 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4476.0:84833 3.0 e78/22 3:01606850:::rbd_object_map.117cbccc292e:head v 78'1246, mlcod=78'1242)", 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:00.729165+0000", 2024-09-22T01:02:04.075 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.2466991350000001, 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050414409999999998, 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.729165+0000", 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.729165+0000", 2024-09-22T01:02:04.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.729166+0000", 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5060000000000001e-06 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.729178+0000", 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1533e-05 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.729203+0000", 2024-09-22T01:02:04.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4802000000000001e-05 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.729217+0000", 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4455e-05 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.729327+0000", 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109881 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.729357+0000", 2024-09-22T01:02:04.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0125999999999999e-05 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.734128+0000", 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004770959 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:00.734206+0000", 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8178999999999998e-05 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.079 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.080 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:04.081 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:04.081 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:04.078+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:04.181 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:04.182+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:04.243 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:04.243 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:04.243 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:04.243 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:04.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:04.282 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:04.279+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:04.383 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:04.383+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:04.418 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:04.418 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:04.418 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:04.418 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:04.418 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:04.432 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:04.484 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:04.482+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:04.560 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:04.560 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:04.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:04.560 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085853027999999, 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:04.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:04.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085832198999999, 2024-09-22T01:02:04.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:04.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:04.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:04.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085803683999998, 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:04.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085781034, 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:04.574 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:04.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:04.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085759038999999, 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.579 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085733975, 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085711541999999, 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085690065000001, 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.592 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:04.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.085661306999999, 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:04.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.084848314999999, 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.084823629999999, 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.605 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:04.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:04.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.080502096, 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:04.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:04.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.080473245, 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:04.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.08045152, 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:04.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.080431090000001, 2024-09-22T01:02:04.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:04.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:04.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.626 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.080408773999999, 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:04.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.080340978999999, 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:04.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.080307842, 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:04.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:04.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.080283919999999, 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:04.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:04.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.080260323000001, 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.645 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:04.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:04.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:04.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:04.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:04.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.649 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:04.649 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:04.649 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:04.650 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:04.651 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:04.650+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:04.751 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:04.750+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:04.767 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:02:04.785 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:04.785 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:04.785 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:04.785 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:04.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:04.852 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:04.850+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:04.920 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:04.920 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:04.920 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:04.920 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:04.920 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:04.934 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:04.952 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:04.950+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:05.053 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:05.050+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.090369703999997, 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.061 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:02:05.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:02:05.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.090317181000003, 2024-09-22T01:02:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:05.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:02:05.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.090264859999998, 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.069 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:02:05.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:02:05.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.090099086000002, 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.073 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.077 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.090070658000002, 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:02:05.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.090042849, 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:05.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:02:05.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.090014744000001, 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089987202000003, 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:05.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089959554000004, 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.097 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089932408999999, 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089904449999999, 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089876635000003, 2024-09-22T01:02:05.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:05.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:02:05.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:02:05.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.109 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089849690000001, 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089821989000001, 2024-09-22T01:02:05.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.117 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089707564999998, 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.121 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089668207000003, 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:05.125 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089638051999998, 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:02:05.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:02:05.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:02:05.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:02:05.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:05.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.130 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.131 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.131 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:05.131 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089608794, 2024-09-22T01:02:05.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:02:05.131 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.131 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.131 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:02:05.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.132 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:02:05.135 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089580918000003, 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.136 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.089551544000003, 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.140 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:05.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:02:05.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:02:05.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.144 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:05.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:05.188 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:05.187+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:05.255 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:05.254+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:05.270 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:05.270 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:05.271 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:05.271 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:05.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:05.372 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:05.371+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:05.449 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:05.449 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:05.449 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:05.449 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:05.449 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:05.456 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:05.454+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:05.463 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:05.500 INFO:teuthology.orchestra.run.smithi002.stdout:4011 punch from 0x2be4ca9 to 0x2be8536, (0x388d bytes) 2024-09-22T01:02:05.515 INFO:teuthology.orchestra.run.smithi002.stdout:4012 write 0xd537ae6 thru 0xd53be16 (0x4331 bytes) 2024-09-22T01:02:05.531 INFO:teuthology.orchestra.run.smithi002.stdout:4013 trunc from 0xd53be17 to 0x68b415b 2024-09-22T01:02:05.557 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:05.554+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:05.618 INFO:teuthology.orchestra.run.smithi002.stdout:4015 write 0xc32096b thru 0xc330332 (0xf9c8 bytes) 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272612216000001, 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272575247000001, 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272546977999999, 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.639 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:02:05.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272509102000001, 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272470512, 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:02:05.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:02:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272398184, 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:02:05.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272360318, 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:05.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:02:05.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:02:05.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272318079, 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272263693999999, 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.663 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:02:05.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272162929, 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.667 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272076004000001, 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:02:05.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.272017679999999, 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.271564529999999, 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.270926043999999, 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:02:05.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.687 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.270638447, 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.691 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.270035478, 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:05.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:02:05.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:05.695 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.268955179000001, 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:02:05.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.268885300999999, 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:02:05.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:02:05.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:02:05.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.26832331, 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.704 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:02:05.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:02:05.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:02:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.267945885, 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:05.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:02:05.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:02:05.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:02:05.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.712 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:05.712 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:05.712 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:05.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:05.714 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:05.711+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:4016 trunc from 0xc330333 to 0x460d7ac 2024-09-22T01:02:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:4017 read 0x3c52392 thru 0x3c5ce00 (0xaa6f bytes) 2024-09-22T01:02:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:4018 punch from 0x22cb92f to 0x22cdbe9, (0x22ba bytes) 2024-09-22T01:02:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:4019 punch from 0xa6b6f3 to 0xa72961, (0x726e bytes) 2024-09-22T01:02:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:4021 write 0x7dc587a thru 0x7dcf47d (0x9c04 bytes) 2024-09-22T01:02:05.715 INFO:teuthology.orchestra.run.smithi002.stdout:4022 read 0x7914281 thru 0x791e68f (0xa40f bytes) 2024-09-22T01:02:05.715 INFO:teuthology.orchestra.run.smithi002.stdout:4023 write 0xdc621e thru 0xdd5947 (0xf72a bytes) 2024-09-22T01:02:05.738 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:05.738 INFO:teuthology.orchestra.run.smithi002.stdout:4024 write 0x96b42ff thru 0x96c35ff (0xf301 bytes) 2024-09-22T01:02:05.746 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:05.746 INFO:teuthology.orchestra.run.smithi002.stdout:4026 write 0xb31d9e thru 0xb3efe9 (0xd24c bytes) 2024-09-22T01:02:05.784 INFO:teuthology.orchestra.run.smithi002.stdout:4027 trunc from 0x96c3600 to 0x3e73de0 2024-09-22T01:02:05.814 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:05.814+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:05.863 INFO:teuthology.orchestra.run.smithi002.stdout:4030 writesame 0xb6ac2d5 thru 0xb6aea88 data_size 0x7(0x27b4 bytes) 2024-09-22T01:02:05.865 INFO:teuthology.orchestra.run.smithi002.stdout:4031 trunc from 0xb6aea89 to 0x9e3c874 2024-09-22T01:02:05.874 INFO:teuthology.orchestra.run.smithi002.stdout:4032 write 0x12f7d65 thru 0x12f81ac (0x448 bytes) 2024-09-22T01:02:05.895 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:05.895 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:05.895 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:05.895 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:05.896 INFO:teuthology.orchestra.run.smithi002.stdout:4033 read 0x7cf02e0 thru 0x7cfd7ac (0xd4cd bytes) 2024-09-22T01:02:05.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:05.910 INFO:teuthology.orchestra.run.smithi002.stdout:4034 write 0xea7d184 thru 0xea85aab (0x8928 bytes) 2024-09-22T01:02:05.915 INFO:teuthology.orchestra.run.smithi002.stdout:4035 read 0x3c562f9 thru 0x3c58d26 (0x2a2e bytes) 2024-09-22T01:02:05.915 INFO:teuthology.orchestra.run.smithi002.stdout:4036 read 0x18f5ddb thru 0x18f87a9 (0x29cf bytes) 2024-09-22T01:02:05.916 INFO:teuthology.orchestra.run.smithi002.stdout:4039 write 0x2ef9a69 thru 0x2efbd6b (0x2303 bytes) 2024-09-22T01:02:05.928 INFO:teuthology.orchestra.run.smithi002.stdout:4040 read 0x66eacd5 thru 0x66f2bd3 (0x7eff bytes) 2024-09-22T01:02:05.928 INFO:teuthology.orchestra.run.smithi002.stdout:4044 writesame 0x78f5c8a thru 0x78fa144 data_size 0x45(0x44bb bytes) 2024-09-22T01:02:05.931 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:05.931 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:05.932 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:05.932 INFO:teuthology.orchestra.run.smithi002.stdout:4047 read 0x4a9b0a0 thru 0x4a9d86c (0x27cd bytes) 2024-09-22T01:02:05.932 INFO:teuthology.orchestra.run.smithi002.stdout:4048 read 0x52dd1e8 thru 0x52e4ebe (0x7cd7 bytes) 2024-09-22T01:02:05.932 INFO:teuthology.orchestra.run.smithi002.stdout:4049 read 0x28a8ad9 thru 0x28b5349 (0xc871 bytes) 2024-09-22T01:02:05.933 INFO:teuthology.orchestra.run.smithi002.stdout:4050 punch from 0x968e15f to 0x9697806, (0x96a7 bytes) 2024-09-22T01:02:05.933 INFO:teuthology.orchestra.run.smithi002.stdout:4051 punch from 0xbb187e7 to 0xbb21b74, (0x938d bytes) 2024-09-22T01:02:05.933 INFO:teuthology.orchestra.run.smithi002.stdout:4053 punch from 0x860a44d to 0x86176ea, (0xd29d bytes) 2024-09-22T01:02:05.933 INFO:teuthology.orchestra.run.smithi002.stdout:4054 write 0x9f439a3 thru 0x9f46a34 (0x3092 bytes) 2024-09-22T01:02:05.949 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:05.949 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:05.955 INFO:teuthology.orchestra.run.smithi002.stdout:4055 trunc from 0xea85aac to 0x14e5fa4 2024-09-22T01:02:05.956 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:05.955+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:05.977 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:05.977 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:05.992 INFO:teuthology.orchestra.run.smithi002.stdout:4056 read 0x60a968 thru 0x615028 (0xa6c1 bytes) 2024-09-22T01:02:05.993 INFO:teuthology.orchestra.run.smithi002.stdout:4057 read 0x9ef2fc thru 0x9f48ea (0x55ef bytes) 2024-09-22T01:02:06.000 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:06.000 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:06.014 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:06.011+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:06.015 INFO:teuthology.orchestra.run.smithi002.stdout:4059 write 0xc1f6dce thru 0xc205308 (0xe53b bytes) 2024-09-22T01:02:06.021 INFO:teuthology.orchestra.run.smithi002.stdout:4060 clone 33 order 24 su 4194304 sc 7 2024-09-22T01:02:06.023 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:06.023 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:06.040 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:06.040 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:06.043 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:06.043 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:06.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:06.062 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:06.062 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:06.080 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:06.080 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:06.140 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:06.139+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:06.214 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:06.214 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:06.216 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:06.214+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:06.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:06.317 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:06.314+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:06.395 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:06.395 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:06.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:06.417 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:06.418+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:06.518 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:06.515+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:06.532 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:06.532 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:06.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:06.632 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:06.631+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:06.713 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:06.713 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:06.720 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:06.718+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:06.727 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:06.796 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:02:06.821 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:06.819+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:06.880 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:06.880 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:06.886 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone32 from 0xc205309 (overlap 0x14e5fa4) to 0x85bfdb 2024-09-22T01:02:06.903 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:06.922 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:06.919+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:07.023 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:07.022+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:07.049 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:07.049 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:07.067 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:07.133 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-22T01:02:07.133 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:07.130+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:07.225 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:07.223+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:07.241 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:07.241 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:07.258 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:07.326 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:07.323+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:07.418 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:07.419 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:07.427 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:07.423+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:07.434 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:07.500 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:07.500 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:07.500 INFO:teuthology.orchestra.run.smithi002.stdout:4066 punch from 0x3df2e7f to 0x3dfbfe9, (0x916a bytes) 2024-09-22T01:02:07.506 INFO:teuthology.orchestra.run.smithi002.stdout:4067 write 0x7f03809 thru 0x7f0a5a6 (0x6d9e bytes) 2024-09-22T01:02:07.513 INFO:teuthology.orchestra.run.smithi002.stdout:4068 read 0xa232123 thru 0xa23a639 (0x8517 bytes) 2024-09-22T01:02:07.514 INFO:teuthology.orchestra.run.smithi002.stdout:4069 write 0x3820c54 thru 0x3830627 (0xf9d4 bytes) 2024-09-22T01:02:07.528 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:07.527+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:07.545 INFO:teuthology.orchestra.run.smithi002.stdout:4070 writesame 0xab1361 thru 0xab6f24 data_size 0x7(0x5bc4 bytes) 2024-09-22T01:02:07.557 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:07.557 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:07.571 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:07.575 INFO:teuthology.orchestra.run.smithi002.stdout:4071 read 0x8c0cf52 thru 0x8c15dd7 (0x8e86 bytes) 2024-09-22T01:02:07.575 INFO:teuthology.orchestra.run.smithi002.stdout:4072 read 0x4cb18ab thru 0x4cbc7fe (0xaf54 bytes) 2024-09-22T01:02:07.575 INFO:teuthology.orchestra.run.smithi002.stdout:4073 read 0x9f2c498 thru 0x9f2f639 (0x31a2 bytes) 2024-09-22T01:02:07.575 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:07.575 INFO:teuthology.orchestra.run.smithi002.stdout:4075 write 0xbc34324 thru 0xbc3cd4b (0x8a28 bytes) 2024-09-22T01:02:07.583 INFO:teuthology.orchestra.run.smithi002.stdout:4076 writesame 0x8fbe029 thru 0x8fcc9b5 data_size 0x1(0xe98d bytes) 2024-09-22T01:02:07.629 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:07.626+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:07.647 INFO:teuthology.orchestra.run.smithi002.stdout:4077 read 0x175e315 thru 0x176ab82 (0xc86e bytes) 2024-09-22T01:02:07.648 INFO:teuthology.orchestra.run.smithi002.stdout:4078 writesame 0x69e7ee thru 0x6a281f data_size 0x53(0x4032 bytes) 2024-09-22T01:02:07.691 INFO:teuthology.orchestra.run.smithi002.stdout:4079 write 0xc64a398 thru 0xc658e3a (0xeaa3 bytes) 2024-09-22T01:02:07.699 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:07.699 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:07.717 INFO:teuthology.orchestra.run.smithi002.stdout:4081 trunc from 0xc658e3b to 0x1bc20fb 2024-09-22T01:02:07.718 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:07.730 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:07.727+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:07.760 INFO:teuthology.orchestra.run.smithi002.stdout:4083 read 0x740148 thru 0x7468ea (0x67a3 bytes) 2024-09-22T01:02:07.762 INFO:teuthology.orchestra.run.smithi002.stdout:4084 read 0x18bf185 thru 0x18c4866 (0x56e2 bytes) 2024-09-22T01:02:07.778 INFO:teuthology.orchestra.run.smithi002.stdout:4085 trunc from 0x1bc20fb to 0xc605974 2024-09-22T01:02:07.784 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:02:07.784 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:02:07.784 INFO:tasks.thrashosds.thrasher:Reweighting osd 0 to 0.9715013319734674 2024-09-22T01:02:07.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 0.9715013319734674 2024-09-22T01:02:07.787 INFO:teuthology.orchestra.run.smithi002.stdout:4086 punch from 0xb03207d to 0xb03ef79, (0xcefc bytes) 2024-09-22T01:02:07.787 INFO:teuthology.orchestra.run.smithi002.stdout:4089 write 0x2614791 thru 0x26160a6 (0x1916 bytes) 2024-09-22T01:02:07.795 INFO:teuthology.orchestra.run.smithi002.stdout:4090 trunc from 0xc605974 to 0x928ffb7 2024-09-22T01:02:07.796 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:02:07.826 INFO:teuthology.orchestra.run.smithi002.stdout:4091 write 0xd61f2e0 thru 0xd6285ab (0x92cc bytes) 2024-09-22T01:02:07.830 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:07.830+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:07.834 INFO:teuthology.orchestra.run.smithi002.stdout:4092 write 0xb7ea66f thru 0xb7f9b3f (0xf4d1 bytes) 2024-09-22T01:02:07.844 INFO:teuthology.orchestra.run.smithi002.stdout:4094 trunc from 0xd6285ac to 0x9d09cda 2024-09-22T01:02:07.856 INFO:teuthology.orchestra.run.smithi002.stdout:4095 read 0x6168736 thru 0x6173d15 (0xb5e0 bytes) 2024-09-22T01:02:07.856 INFO:teuthology.orchestra.run.smithi002.stdout:4096 write 0x606fe9b thru 0x607e76c (0xe8d2 bytes) 2024-09-22T01:02:07.858 INFO:teuthology.orchestra.run.smithi002.stdout:4098 write 0x7c47e2d thru 0x7c496b0 (0x1884 bytes) 2024-09-22T01:02:07.861 INFO:teuthology.orchestra.run.smithi002.stdout:4099 read 0x82b9405 thru 0x82bf104 (0x5d00 bytes) 2024-09-22T01:02:07.862 INFO:teuthology.orchestra.run.smithi002.stdout:4100 read 0x1e306e7 thru 0x1e38cce (0x85e8 bytes) 2024-09-22T01:02:07.862 INFO:teuthology.orchestra.run.smithi002.stdout:4101 write 0x60f66c3 thru 0x6100e99 (0xa7d7 bytes) 2024-09-22T01:02:07.866 INFO:teuthology.orchestra.run.smithi002.stdout:4103 trunc from 0x9d09cda to 0xb69d260 2024-09-22T01:02:07.873 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:07.886 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:07.886 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:07.887 INFO:teuthology.orchestra.run.smithi002.stdout:4105 write 0xe44da06 thru 0xe45d59a (0xfb95 bytes) 2024-09-22T01:02:07.894 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:07.894 INFO:teuthology.orchestra.run.smithi002.stdout:4108 read 0xbd75aa1 thru 0xbd76e7c (0x13dc bytes) 2024-09-22T01:02:07.894 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:07.894 INFO:teuthology.orchestra.run.smithi002.stdout:4110 read 0x570ae69 thru 0x5719fed (0xf185 bytes) 2024-09-22T01:02:07.894 INFO:teuthology.orchestra.run.smithi002.stdout:4111 clone 34 order 25 su 262144 sc 2 2024-09-22T01:02:07.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:07.942 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:07.942+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:08.032 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:08.031+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:08.052 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:08.052 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:08.067 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:08.097 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:08.162 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:08.162+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:08.186 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:08.186 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:08.200 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:08.234 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:08.231+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:08.249 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:08.250 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:08.266 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:08.267 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:08.290 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:08.290 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:08.307 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:08.308 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:08.328 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:08.328 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:08.335 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:08.334+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:08.349 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:08.350 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:08.362 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:08.362 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:08.371 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:08.371 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:08.376 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:08.393 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:08.393 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:08.436 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:08.435+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:08.527 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:08.528 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:08.537 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:08.535+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:08.542 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:08.638 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:08.638+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:08.665 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:08.665 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189398954000001, 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189378125000001, 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:08.670 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189349610000001, 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189326959999999, 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:08.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:08.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:08.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189304965000002, 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:08.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189279900999999, 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:08.687 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.691 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189257468000001, 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189235991, 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:08.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:08.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.189207233000001, 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.188394241000001, 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:08.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:08.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:08.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.188369556000001, 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:08.708 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:08.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.184048021999999, 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.184019170999999, 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.717 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:08.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.183997445999999, 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.183977016, 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:08.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:08.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:08.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.183954700000001, 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.183886905000001, 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.183853767999999, 2024-09-22T01:02:08.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.183829845999998, 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:08.742 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.183806249, 2024-09-22T01:02:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:08.750 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:08.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:08.752 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:08.751+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 0.971501 (f8b4) 2024-09-22T01:02:08.853 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:08.850+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:08.885 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:08.885 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:08.885 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:08.885 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:08.900 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:08.928 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone33 from 0xe45d59b (overlap 0x1bc20fb) to 0x853d18 2024-09-22T01:02:08.953 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:08.951+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:09.024 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:09.025 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:09.039 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:09.080 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:09.079+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:09.114 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-22T01:02:09.153 INFO:teuthology.orchestra.run.smithi002.stdout:4112 writesame 0x21d2bda thru 0x21d6da3 data_size 0x15(0x41ca bytes) 2024-09-22T01:02:09.155 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:09.154+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:09.228 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.257349271000002, 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:02:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:02:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.257296748000002, 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:02:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:02:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.257244427000003, 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:02:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.257078653000001, 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:02:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:02:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.257050225, 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:02:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.257022415999998, 2024-09-22T01:02:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:02:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:02:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256994311, 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:02:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:02:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256966769000002, 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:02:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256939121000002, 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:02:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:02:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256911975999998, 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256884016999997, 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256856202000002, 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256829257, 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256801555999999, 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:02:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256687132000003, 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256647774000001, 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256617618999996, 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:02:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256588360999999, 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256560485000001, 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.256531111000001, 2024-09-22T01:02:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:02:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:02:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.303 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:09.304 INFO:teuthology.orchestra.run.smithi002.stdout:4113 write 0x9fa7eb3 thru 0x9fad39c (0x54ea bytes) 2024-09-22T01:02:09.304 INFO:teuthology.orchestra.run.smithi002.stdout:4115 trunc from 0xe45d59b to 0x9e1c79 2024-09-22T01:02:09.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:09.305 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:09.302+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:09.378 INFO:teuthology.orchestra.run.smithi002.stdout:4116 read 0x57db9b thru 0x583c1e (0x6084 bytes) 2024-09-22T01:02:09.405 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:09.402+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:09.421 INFO:teuthology.orchestra.run.smithi002.stdout:4117 write 0xa6c7a82 thru 0xa6ce190 (0x670f bytes) 2024-09-22T01:02:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:4120 write 0x3fcd9d1 thru 0x3fd4628 (0x6c58 bytes) 2024-09-22T01:02:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:4121 punch from 0x65f5ea0 to 0x65fcdbf, (0x6f1f bytes) 2024-09-22T01:02:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:4122 write 0xa14551b thru 0xa150e5e (0xb944 bytes) 2024-09-22T01:02:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:09.435 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:09.435 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:09.443 INFO:teuthology.orchestra.run.smithi002.stdout:4123 trunc from 0xa6ce191 to 0xceec102 2024-09-22T01:02:09.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:09.458 INFO:teuthology.orchestra.run.smithi002.stdout:4124 write 0x125d143 thru 0x12638a9 (0x6767 bytes) 2024-09-22T01:02:09.460 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:09.461 INFO:teuthology.orchestra.run.smithi002.stdout:4126 read 0x4829438 thru 0x482a422 (0xfeb bytes) 2024-09-22T01:02:09.461 INFO:teuthology.orchestra.run.smithi002.stdout:4127 write 0x752e98 thru 0x75334a (0x4b3 bytes) 2024-09-22T01:02:09.463 INFO:teuthology.orchestra.run.smithi002.stdout:4129 write 0xac7558c thru 0xac7cfd2 (0x7a47 bytes) 2024-09-22T01:02:09.467 INFO:teuthology.orchestra.run.smithi002.stdout:4130 write 0x41ebc6 thru 0x420517 (0x1952 bytes) 2024-09-22T01:02:09.473 INFO:teuthology.orchestra.run.smithi002.stdout:4131 trunc from 0xceec102 to 0xc5884f2 2024-09-22T01:02:09.500 INFO:teuthology.orchestra.run.smithi002.stdout:4132 write 0xc804f92 thru 0xc813041 (0xe0b0 bytes) 2024-09-22T01:02:09.506 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:09.506+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:09.508 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:09.508 INFO:teuthology.orchestra.run.smithi002.stdout:4136 flatten 2024-09-22T01:02:09.523 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:09.523 INFO:teuthology.orchestra.run.smithi002.stdout:4138 punch from 0x2c64db6 to 0x2c73eed, (0xf137 bytes) 2024-09-22T01:02:09.523 INFO:teuthology.orchestra.run.smithi002.stdout:4139 punch from 0x93cee88 to 0x93d2bf9, (0x3d71 bytes) 2024-09-22T01:02:09.523 INFO:teuthology.orchestra.run.smithi002.stdout:4140 write 0x6245e23 thru 0x62523f9 (0xc5d7 bytes) 2024-09-22T01:02:09.530 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:09.530 INFO:teuthology.orchestra.run.smithi002.stdout:4142 write 0x89859ad thru 0x8987470 (0x1ac4 bytes) 2024-09-22T01:02:09.537 INFO:teuthology.orchestra.run.smithi002.stdout:4143 writesame 0x4805661 thru 0x4813bf6 data_size 0x1(0xe596 bytes) 2024-09-22T01:02:09.573 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:09.573 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:09.573 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:09.573 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:09.573 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:09.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:09.632 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:09.631+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:09.635 INFO:teuthology.orchestra.run.smithi002.stdout:4144 write 0xa73f3a6 thru 0xa748267 (0x8ec2 bytes) 2024-09-22T01:02:09.644 INFO:teuthology.orchestra.run.smithi002.stdout:4146 trunc from 0xc813042 to 0xae93bd3 2024-09-22T01:02:09.668 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:09.669 INFO:teuthology.orchestra.run.smithi002.stdout:4148 clone 35 order 24 su 131072 sc 4 2024-09-22T01:02:09.708 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:09.706+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:09.714 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:09.714 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.3563568, 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:02:09.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.356319831, 2024-09-22T01:02:09.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:02:09.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:02:09.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.723 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.356291561999999, 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:09.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:02:09.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.356253686000001, 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.728 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:09.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:02:09.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:02:09.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.356215096, 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:02:09.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:02:09.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:09.736 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.356142768, 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:02:09.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.356104902, 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.741 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:02:09.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.745 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.356062662999999, 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:02:09.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.356008278000001, 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.355907513, 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:02:09.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.355820588, 2024-09-22T01:02:09.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:02:09.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.355762263999999, 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:02:09.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.355309114000001, 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:02:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:02:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.354670627999999, 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.354383030999999, 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.353780062, 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:02:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:02:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:02:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.352699763, 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.352629885000001, 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:02:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.352067893999999, 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.351690468999999, 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:02:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:02:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:09.799 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:09.800 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:09.808 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:09.806+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:09.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:02:09.909 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:09.907+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:09.972 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:09.972 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:09.972 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:09.972 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:09.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:10.028 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:10.027+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:10.083 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-22T01:02:10.107 INFO:teuthology.orchestra.run.smithi002.stdout:4149 write 0x703d777 thru 0x704bd81 (0xe60b bytes) 2024-09-22T01:02:10.111 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:10.107+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:4151 write 0xa037072 thru 0xa040b5a (0x9ae9 bytes) 2024-09-22T01:02:10.155 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:10.156 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:10.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:4155 read 0x4e6b91c thru 0x4e72326 (0x6a0b bytes) 2024-09-22T01:02:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:4156 writesame 0xeba0067 thru 0xebac1d3 data_size 0xd(0xc16d bytes) 2024-09-22T01:02:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:4158 write 0x1bdd4fa thru 0x1be41c0 (0x6cc7 bytes) 2024-09-22T01:02:10.211 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:10.210+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:10.312 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:10.311+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:10.314 INFO:teuthology.orchestra.run.smithi002.stdout:4159 write 0x9a6f22e thru 0x9a6f4f1 (0x2c4 bytes) 2024-09-22T01:02:10.358 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:10.358 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:10.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:10.358 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:10.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.358 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:02:10.358 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 98.464665823999994, 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:02:10.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080668496999998, 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.365 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:10.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:02:10.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.369 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080635192000003, 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:10.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.373 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080538679, 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:02:10.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080465427999997, 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:10.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080400247, 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:02:10.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:02:10.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:02:10.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080330126, 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:02:10.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080296761, 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:02:10.391 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:02:10.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:02:10.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:10.395 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080266338999998, 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:02:10.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080236032999998, 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:10.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.403 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080205630999998, 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:02:10.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080174831999997, 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:02:10.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:02:10.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:02:10.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:02:10.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:02:10.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:02:10.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080144359000002, 2024-09-22T01:02:10.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:02:10.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:02:10.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080113767, 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:02:10.417 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:02:10.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:02:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080075964999999, 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:02:10.421 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:02:10.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080043600000003, 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.080013280999999, 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:10.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.079983388999999, 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.079953629999999, 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.079903164999998, 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:10.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:02:10.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:10.446 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:10.447 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:10.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:10.449 INFO:teuthology.orchestra.run.smithi002.stdout:4160 writesame 0xb96bc09 thru 0xb9709ca data_size 0x25(0x4dc2 bytes) 2024-09-22T01:02:10.449 INFO:teuthology.orchestra.run.smithi002.stdout:4161 write 0xafc3c9a thru 0xafc8563 (0x48ca bytes) 2024-09-22T01:02:10.449 INFO:teuthology.orchestra.run.smithi002.stdout:4162 write 0x4bb8480 thru 0x4bbd385 (0x4f06 bytes) 2024-09-22T01:02:10.450 INFO:teuthology.orchestra.run.smithi002.stdout:4163 read 0x5d25872 thru 0x5d2d474 (0x7c03 bytes) 2024-09-22T01:02:10.450 INFO:teuthology.orchestra.run.smithi002.stdout:4164 read 0xbbcd714 thru 0xbbda711 (0xcffe bytes) 2024-09-22T01:02:10.450 INFO:teuthology.orchestra.run.smithi002.stdout:4166 read 0xc1b962 thru 0xc1ba96 (0x135 bytes) 2024-09-22T01:02:10.450 INFO:teuthology.orchestra.run.smithi002.stdout:4167 read 0xac9476b thru 0xac95aac (0x1342 bytes) 2024-09-22T01:02:10.450 INFO:teuthology.orchestra.run.smithi002.stdout:4168 write 0x18d4d9b thru 0x18e0793 (0xb9f9 bytes) 2024-09-22T01:02:10.450 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.451 INFO:teuthology.orchestra.run.smithi002.stdout:4170 read 0xd6f46ed thru 0xd703d84 (0xf698 bytes) 2024-09-22T01:02:10.451 INFO:teuthology.orchestra.run.smithi002.stdout:4172 flatten 2024-09-22T01:02:10.451 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:10.446+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:10.521 INFO:teuthology.orchestra.run.smithi002.stdout:4173 trunc from 0xebac1d4 to 0xd5294c1 2024-09-22T01:02:10.529 INFO:teuthology.orchestra.run.smithi002.stdout:4175 read 0x5019399 thru 0x50257b4 (0xc41c bytes) 2024-09-22T01:02:10.530 INFO:teuthology.orchestra.run.smithi002.stdout:4176 write 0xc5ebf thru 0xd2e34 (0xcf76 bytes) 2024-09-22T01:02:10.533 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.544 INFO:teuthology.orchestra.run.smithi002.stdout:4178 trunc from 0xd5294c1 to 0x7be5593 2024-09-22T01:02:10.549 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:10.547+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:10.576 INFO:teuthology.orchestra.run.smithi002.stdout:4179 write 0xbc5d248 thru 0xbc63b69 (0x6922 bytes) 2024-09-22T01:02:10.581 INFO:teuthology.orchestra.run.smithi002.stdout:4180 punch from 0x3a4fc8c to 0x3a5dbe0, (0xdf54 bytes) 2024-09-22T01:02:10.582 INFO:teuthology.orchestra.run.smithi002.stdout:4181 punch from 0x5b7ea6c to 0x5b8ad99, (0xc32d bytes) 2024-09-22T01:02:10.582 INFO:teuthology.orchestra.run.smithi002.stdout:4183 read 0x377fa8d thru 0x378ec85 (0xf1f9 bytes) 2024-09-22T01:02:10.582 INFO:teuthology.orchestra.run.smithi002.stdout:4184 punch from 0x475928a to 0x476532a, (0xc0a0 bytes) 2024-09-22T01:02:10.582 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.582 INFO:teuthology.orchestra.run.smithi002.stdout:4187 read 0x2e886c2 thru 0x2e8c71b (0x405a bytes) 2024-09-22T01:02:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:4188 write 0xe9da5c5 thru 0xe9e6635 (0xc071 bytes) 2024-09-22T01:02:10.599 INFO:teuthology.orchestra.run.smithi002.stdout:4189 read 0xc518187 thru 0xc51afac (0x2e26 bytes) 2024-09-22T01:02:10.599 INFO:teuthology.orchestra.run.smithi002.stdout:4190 writesame 0x944e20a thru 0x9458bf8 data_size 0x11(0xa9ef bytes) 2024-09-22T01:02:10.602 INFO:teuthology.orchestra.run.smithi002.stdout:4191 punch from 0x91f8ba4 to 0x91fbb1e, (0x2f7a bytes) 2024-09-22T01:02:10.609 INFO:teuthology.orchestra.run.smithi002.stdout:4192 write 0xeb3d754 thru 0xeb45827 (0x80d4 bytes) 2024-09-22T01:02:10.613 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.614 INFO:teuthology.orchestra.run.smithi002.stdout:4195 writesame 0x4146f25 thru 0x414fd8f data_size 0x3(0x8e6b bytes) 2024-09-22T01:02:10.619 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:10.619 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:10.625 INFO:teuthology.orchestra.run.smithi002.stdout:4196 read 0xd1e869b thru 0xd1ef4d9 (0x6e3f bytes) 2024-09-22T01:02:10.626 INFO:teuthology.orchestra.run.smithi002.stdout:4197 punch from 0x6f882de to 0x6f9252c, (0xa24e bytes) 2024-09-22T01:02:10.626 INFO:teuthology.orchestra.run.smithi002.stdout:4198 punch from 0x4f9aee0 to 0x4fa3114, (0x8234 bytes) 2024-09-22T01:02:10.626 INFO:teuthology.orchestra.run.smithi002.stdout:4200 read 0x50f2647 thru 0x50f7cbb (0x5675 bytes) 2024-09-22T01:02:10.626 INFO:teuthology.orchestra.run.smithi002.stdout:4202 read 0x8347f3d thru 0x834ac75 (0x2d39 bytes) 2024-09-22T01:02:10.626 INFO:teuthology.orchestra.run.smithi002.stdout:4203 punch from 0xe9134f3 to 0xe919864, (0x6371 bytes) 2024-09-22T01:02:10.627 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.627 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.627 INFO:teuthology.orchestra.run.smithi002.stdout:4206 punch from 0x3999293 to 0x39a1e68, (0x8bd5 bytes) 2024-09-22T01:02:10.627 INFO:teuthology.orchestra.run.smithi002.stdout:4208 write 0x88f22d1 thru 0x88fcff0 (0xad20 bytes) 2024-09-22T01:02:10.628 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.628 INFO:teuthology.orchestra.run.smithi002.stdout:4212 write 0x92e8c2a thru 0x92ec73c (0x3b13 bytes) 2024-09-22T01:02:10.628 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:10.628 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:10.629 INFO:teuthology.orchestra.run.smithi002.stdout:4213 read 0x8ebd08b thru 0x8ec1d09 (0x4c7f bytes) 2024-09-22T01:02:10.630 INFO:teuthology.orchestra.run.smithi002.stdout:4214 writesame 0x9ecd182 thru 0x9ed28b2 data_size 0x11(0x5731 bytes) 2024-09-22T01:02:10.633 INFO:teuthology.orchestra.run.smithi002.stdout:4217 punch from 0x75a0401 to 0x75a4922, (0x4521 bytes) 2024-09-22T01:02:10.634 INFO:teuthology.orchestra.run.smithi002.stdout:4218 read 0x84f5f78 thru 0x84ffa8d (0x9b16 bytes) 2024-09-22T01:02:10.634 INFO:teuthology.orchestra.run.smithi002.stdout:4219 trunc from 0xeb45828 to 0xe9be4ef 2024-09-22T01:02:10.638 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:10.639 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:10.642 INFO:teuthology.orchestra.run.smithi002.stdout:4221 write 0x8019712 thru 0x8023972 (0xa261 bytes) 2024-09-22T01:02:10.644 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:10.646 INFO:teuthology.orchestra.run.smithi002.stdout:4222 read 0x281b94 thru 0x290b00 (0xef6d bytes) 2024-09-22T01:02:10.647 INFO:teuthology.orchestra.run.smithi002.stdout:4223 write 0x69397a2 thru 0x693ebc5 (0x5424 bytes) 2024-09-22T01:02:10.649 INFO:teuthology.orchestra.run.smithi002.stdout:4224 punch from 0xe10fbb0 to 0xe11e4de, (0xe92e bytes) 2024-09-22T01:02:10.649 INFO:teuthology.orchestra.run.smithi002.stdout:4225 write 0xb17c23f thru 0xb17e671 (0x2433 bytes) 2024-09-22T01:02:10.649 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:10.646+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:10.650 INFO:teuthology.orchestra.run.smithi002.stdout:4228 writesame 0xae945ea thru 0xae9ae05 data_size 0xc(0x681c bytes) 2024-09-22T01:02:10.652 INFO:teuthology.orchestra.run.smithi002.stdout:4229 read 0xa4e5295 thru 0xa4f4ff8 (0xfd64 bytes) 2024-09-22T01:02:10.653 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.653 INFO:teuthology.orchestra.run.smithi002.stdout:4232 trunc from 0xe9be4ef to 0xec08b2e 2024-09-22T01:02:10.658 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:10.659 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:10.661 INFO:teuthology.orchestra.run.smithi002.stdout:4233 read 0x316bd6e thru 0x3172f3a (0x71cd bytes) 2024-09-22T01:02:10.661 INFO:teuthology.orchestra.run.smithi002.stdout:4234 writesame 0x695d60 thru 0x6a33b7 data_size 0x4c(0xd658 bytes) 2024-09-22T01:02:10.664 INFO:teuthology.orchestra.run.smithi002.stdout:4235 write 0x9bcc575 thru 0x9bdaf0c (0xe998 bytes) 2024-09-22T01:02:10.666 INFO:teuthology.orchestra.run.smithi002.stdout:4237 write 0xbe034c7 thru 0xbe05de0 (0x291a bytes) 2024-09-22T01:02:10.668 INFO:teuthology.orchestra.run.smithi002.stdout:4238 writesame 0x7597692 thru 0x759c396 data_size 0x1(0x4d05 bytes) 2024-09-22T01:02:10.677 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:10.677 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:10.691 INFO:teuthology.orchestra.run.smithi002.stdout:4240 writesame 0x7efbe3 thru 0x7f4d5c data_size 0x1(0x517a bytes) 2024-09-22T01:02:10.695 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:10.704 INFO:teuthology.orchestra.run.smithi002.stdout:4248 read 0x8a59480 thru 0x8a64c57 (0xb7d8 bytes) 2024-09-22T01:02:10.705 INFO:teuthology.orchestra.run.smithi002.stdout:4249 punch from 0x25198e3 to 0x251dbfe, (0x431b bytes) 2024-09-22T01:02:10.705 INFO:teuthology.orchestra.run.smithi002.stdout:4251 read 0x1e19e7 thru 0x1e78da (0x5ef4 bytes) 2024-09-22T01:02:10.705 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.705 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.705 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.705 INFO:teuthology.orchestra.run.smithi002.stdout:4256 write 0x4613aec thru 0x461f5ef (0xbb04 bytes) 2024-09-22T01:02:10.708 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.708 INFO:teuthology.orchestra.run.smithi002.stdout:4259 read 0xd8bf66 thru 0xd90022 (0x40bd bytes) 2024-09-22T01:02:10.708 INFO:teuthology.orchestra.run.smithi002.stdout:4260 write 0x51b873e thru 0x51b878f (0x52 bytes) 2024-09-22T01:02:10.709 INFO:teuthology.orchestra.run.smithi002.stdout:4261 write 0x5f6ffdd thru 0x5f7cbde (0xcc02 bytes) 2024-09-22T01:02:10.714 INFO:teuthology.orchestra.run.smithi002.stdout:4263 write 0xe3ec098 thru 0xe3f27da (0x6743 bytes) 2024-09-22T01:02:10.715 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:10.716 INFO:teuthology.orchestra.run.smithi002.stdout:4265 read 0x7582320 thru 0x7584112 (0x1df3 bytes) 2024-09-22T01:02:10.717 INFO:teuthology.orchestra.run.smithi002.stdout:4266 trunc from 0xec08b2e to 0xe40752a 2024-09-22T01:02:10.725 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.732 INFO:teuthology.orchestra.run.smithi002.stdout:4268 trunc from 0xe40752a to 0x96fa7d2 2024-09-22T01:02:10.739 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:10.739 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:10.746 INFO:teuthology.orchestra.run.smithi002.stdout:4269 read 0x3440dc8 thru 0x344a3c6 (0x95ff bytes) 2024-09-22T01:02:10.747 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.747 INFO:teuthology.orchestra.run.smithi002.stdout:4271 writesame 0x6fbc26d thru 0x6fcbebf data_size 0x1(0xfc53 bytes) 2024-09-22T01:02:10.750 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:10.747+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:10.759 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:10.759 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:10.790 INFO:teuthology.orchestra.run.smithi002.stdout:4273 write 0xfb13a8 thru 0xfbd9a6 (0xc5ff bytes) 2024-09-22T01:02:10.793 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.793 INFO:teuthology.orchestra.run.smithi002.stdout:4275 punch from 0x5709903 to 0x5711138, (0x7835 bytes) 2024-09-22T01:02:10.793 INFO:teuthology.orchestra.run.smithi002.stdout:4276 read 0x67e41bb thru 0x67e4630 (0x476 bytes) 2024-09-22T01:02:10.793 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.793 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:10.794 INFO:teuthology.orchestra.run.smithi002.stdout:4280 punch from 0x5c2a6da to 0x5c3150d, (0x6e33 bytes) 2024-09-22T01:02:10.794 INFO:teuthology.orchestra.run.smithi002.stdout:4281 write 0x2faa02f thru 0x2fb2ea1 (0x8e73 bytes) 2024-09-22T01:02:10.794 INFO:teuthology.orchestra.run.smithi002.stdout:4282 read 0x76eb7e5 thru 0x76efe78 (0x4694 bytes) 2024-09-22T01:02:10.795 INFO:teuthology.orchestra.run.smithi002.stdout:4283 write 0x59f8761 thru 0x5a0098d (0x822d bytes) 2024-09-22T01:02:10.798 INFO:teuthology.orchestra.run.smithi002.stdout:4284 read 0x32a2963 thru 0x32afcd9 (0xd377 bytes) 2024-09-22T01:02:10.810 INFO:teuthology.orchestra.run.smithi002.stdout:4286 writesame 0x98c74d8 thru 0x98d2e9a data_size 0x1(0xb9c3 bytes) 2024-09-22T01:02:10.812 INFO:teuthology.orchestra.run.smithi002.stdout:4287 read 0x8173e72 thru 0x81776bb (0x384a bytes) 2024-09-22T01:02:10.813 INFO:teuthology.orchestra.run.smithi002.stdout:4288 write 0x39c0eff thru 0x39cdb68 (0xcc6a bytes) 2024-09-22T01:02:10.816 INFO:teuthology.orchestra.run.smithi002.stdout:4289 punch from 0x7746c90 to 0x7753974, (0xcce4 bytes) 2024-09-22T01:02:10.822 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:02:10.831 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:10.831 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:10.834 INFO:teuthology.orchestra.run.smithi002.stdout:4291 write 0xcabc74e thru 0xcacb187 (0xea3a bytes) 2024-09-22T01:02:10.842 INFO:teuthology.orchestra.run.smithi002.stdout:4292 write 0x6b12003 thru 0x6b12731 (0x72f bytes) 2024-09-22T01:02:10.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:10.869 INFO:teuthology.orchestra.run.smithi002.stdout:4293 trunc from 0xcacb188 to 0x2921e4 2024-09-22T01:02:10.888 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:10.887+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:10.899 INFO:teuthology.orchestra.run.smithi002.stdout:4294 trunc from 0x2921e4 to 0x1ee98e2 2024-09-22T01:02:10.907 INFO:teuthology.orchestra.run.smithi002.stdout:4295 read 0x144f5a4 thru 0x145ca78 (0xd4d5 bytes) 2024-09-22T01:02:10.932 INFO:teuthology.orchestra.run.smithi002.stdout:4296 write 0x95c3531 thru 0x95c4560 (0x1030 bytes) 2024-09-22T01:02:10.936 INFO:teuthology.orchestra.run.smithi002.stdout:4297 read 0x20928cf thru 0x20937b6 (0xee8 bytes) 2024-09-22T01:02:10.951 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:10.947+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:4298 write 0x9dc7c14 thru 0x9dcb195 (0x3582 bytes) 2024-09-22T01:02:10.955 INFO:teuthology.orchestra.run.smithi002.stdout:4299 write 0x50a0e0a thru 0x50a4bd9 (0x3dd0 bytes) 2024-09-22T01:02:10.961 INFO:teuthology.orchestra.run.smithi002.stdout:4300 write 0x47c2df7 thru 0x47c65a1 (0x37ab bytes) 2024-09-22T01:02:10.966 INFO:teuthology.orchestra.run.smithi002.stdout:4301 read 0x6572905 thru 0x6574c56 (0x2352 bytes) 2024-09-22T01:02:10.985 INFO:teuthology.orchestra.run.smithi002.stdout:4302 write 0xb87b9ea thru 0xb87c8d4 (0xeeb bytes) 2024-09-22T01:02:10.992 INFO:teuthology.orchestra.run.smithi002.stdout:4303 writesame 0x499946a thru 0x499b597 data_size 0x1f(0x212e bytes) 2024-09-22T01:02:10.997 INFO:teuthology.orchestra.run.smithi002.stdout:4304 writesame 0xa46a7c6 thru 0xa470bf6 data_size 0x1(0x6431 bytes) 2024-09-22T01:02:11.010 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:11.010 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:11.025 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:11.033 INFO:teuthology.orchestra.run.smithi002.stdout:4305 write 0x5ff9bd thru 0x60086f (0xeb3 bytes) 2024-09-22T01:02:11.036 INFO:teuthology.orchestra.run.smithi002.stdout:4307 write 0x4938173 thru 0x493a3b9 (0x2247 bytes) 2024-09-22T01:02:11.039 INFO:teuthology.orchestra.run.smithi002.stdout:4309 punch from 0x7f9087f to 0x7f9e251, (0xd9d2 bytes) 2024-09-22T01:02:11.039 INFO:teuthology.orchestra.run.smithi002.stdout:4311 write 0x94f2db1 thru 0x94fe4be (0xb70e bytes) 2024-09-22T01:02:11.042 INFO:teuthology.orchestra.run.smithi002.stdout:4312 read 0x98f88f thru 0x999397 (0x9b09 bytes) 2024-09-22T01:02:11.043 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:11.054 INFO:teuthology.orchestra.run.smithi002.stdout:4314 trunc from 0xb87c8d5 to 0x6aa2a9a 2024-09-22T01:02:11.068 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:11.067+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:11.072 INFO:teuthology.orchestra.run.smithi002.stdout:4316 read 0x1c9a8d5 thru 0x1ca3f37 (0x9663 bytes) 2024-09-22T01:02:11.072 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:11.081 INFO:teuthology.orchestra.run.smithi002.stdout:4318 write 0x6eadb61 thru 0x6eb0bc1 (0x3061 bytes) 2024-09-22T01:02:11.084 INFO:teuthology.orchestra.run.smithi002.stdout:4319 read 0x1991457 thru 0x199daee (0xc698 bytes) 2024-09-22T01:02:11.084 INFO:teuthology.orchestra.run.smithi002.stdout:4320 read 0x5fe1428 thru 0x5fe9219 (0x7df2 bytes) 2024-09-22T01:02:11.085 INFO:teuthology.orchestra.run.smithi002.stdout:4321 write 0x54ec677 thru 0x54f0bfa (0x4584 bytes) 2024-09-22T01:02:11.089 INFO:teuthology.orchestra.run.smithi002.stdout:4322 punch from 0x1ab251c to 0x1ac14a4, (0xef88 bytes) 2024-09-22T01:02:11.089 INFO:teuthology.orchestra.run.smithi002.stdout:4323 read 0x5aeb83b thru 0x5af6f37 (0xb6fd bytes) 2024-09-22T01:02:11.108 INFO:teuthology.orchestra.run.smithi002.stdout:4324 write 0xd43c06b thru 0xd444d30 (0x8cc6 bytes) 2024-09-22T01:02:11.115 INFO:teuthology.orchestra.run.smithi002.stdout:4325 clone 36 order 19 su 8192 sc 8 2024-09-22T01:02:11.153 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:11.151+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:11.166 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:11.166 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:11.180 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:11.270 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:11.270+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:11.347 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:11.347 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:11.354 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:11.351+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:11.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:11.455 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:11.454+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:11.530 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:11.530 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:11.544 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:11.556 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:11.555+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:11.657 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:11.655+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:11.714 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:11.714 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:11.729 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:11.757 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:11.755+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:11.858 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:11.855+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:11.864 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:11.864 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:11.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:11.959 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:11.955+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:12.008 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:12.009 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:12.026 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:12.070 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:12.070+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:12.092 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-22T01:02:12.159 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:12.159 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:12.161 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:12.159+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:12.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:12.266 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:12.266+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:12.347 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:12.347 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:12.363 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:12.364 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:12.359+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:12.422 INFO:teuthology.orchestra.run.smithi002.stdout:4327 trunc from 0xd444d31 to 0x11e3d5d 2024-09-22T01:02:12.464 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:12.463+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:12.531 INFO:teuthology.orchestra.run.smithi002.stdout:4328 write 0x7994948 thru 0x79a42a5 (0xf95e bytes) 2024-09-22T01:02:12.554 INFO:teuthology.orchestra.run.smithi002.stdout:4329 clone 37 order 23 su 524288 sc 3 2024-09-22T01:02:12.558 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:12.558 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:12.565 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:12.566+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:12.573 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:12.667 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:12.663+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:12.740 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:12.740 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:12.755 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:12.767 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:12.763+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:12.776 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:12.848 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:02:12.886 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:12.886 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:12.886 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:12.886+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:12.906 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:12.937 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone36 2024-09-22T01:02:12.968 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:12.963+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:12.978 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:13.020 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:13.020 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:13.061 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:13.062 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:13.069 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:13.067+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:13.092 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:13.092 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:13.128 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:13.129 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:13.133 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:13.133 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:13.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:13.133 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:13.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658553435999998, 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:13.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658532607000001, 2024-09-22T01:02:13.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:13.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:13.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658504092000001, 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:13.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658481441999999, 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:13.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658459446999998, 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658434382999999, 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658411950000001, 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658390473000001, 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.165 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.169 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.658361715000002, 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.173 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.657548723000001, 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:13.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.178 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.657524037999998, 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:13.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.653202503999999, 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:13.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.653173653, 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.653151928, 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:13.192 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:13.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.653131498, 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:13.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:13.201 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.653109182000001, 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:13.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.653041387000002, 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:13.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.653008249999999, 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:13.210 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:13.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.652984327999999, 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.215 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.652960731, 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:13.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:13.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:13.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:13.223 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:13.224 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:13.225 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:13.225 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:13.225 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:13.226 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:13.226 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:13.226 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:13.227 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:13.223+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-22T01:02:13.325 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:13.322+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:13.357 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:13.357 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:13.357 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:13.357 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:13.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:13.426 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:13.423+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:13.494 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:13.495 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:13.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:13.526 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:13.526+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:13.627 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:13.623+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:4330 writesame 0x2b804c0 thru 0x2b8065c data_size 0x1(0x19d bytes) 2024-09-22T01:02:13.663 INFO:teuthology.orchestra.run.smithi002.stdout:4331 write 0x1d0d5c3 thru 0x1d11317 (0x3d55 bytes) 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708510851, 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.678 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:02:13.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708458327999999, 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.683 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:02:13.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:02:13.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708406007000001, 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:02:13.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.690 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708240232999998, 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708211804999998, 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:02:13.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.698 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708183996000002, 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:02:13.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708155890999997, 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.703 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:02:13.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708128348999999, 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.707 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:02:13.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708100700999999, 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:02:13.711 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708073556000002, 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:02:13.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:02:13.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708045597000002, 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.723 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:13.724 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.708017781999999, 2024-09-22T01:02:13.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:02:13.724 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.724 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.724 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:02:13.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:13.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.707990836999997, 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:02:13.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.707963136000004, 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.733 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:13.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.707848712000001, 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.737 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:02:13.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:02:13.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.707809353999998, 2024-09-22T01:02:13.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:13.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:13.745 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.707779199000001, 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:02:13.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.707749941000003, 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.750 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.707722064999999, 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.754 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:02:13.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.707692690999998, 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:13.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:13.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:13.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:13.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:13.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:02:13.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:02:13.762 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.762 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.762 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.762 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:13.762 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:13.762 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:13.764 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:13.765 INFO:teuthology.orchestra.run.smithi002.stdout:4332 writesame 0x8039a69 thru 0x804763f data_size 0x1(0xdbd7 bytes) 2024-09-22T01:02:13.765 INFO:teuthology.orchestra.run.smithi002.stdout:4333 writesame 0x4689ad6 thru 0x46964cf data_size 0x2(0xc9fa bytes) 2024-09-22T01:02:13.765 INFO:teuthology.orchestra.run.smithi002.stdout:4334 punch from 0x4dd8cd4 to 0x4dda8f9, (0x1c25 bytes) 2024-09-22T01:02:13.765 INFO:teuthology.orchestra.run.smithi002.stdout:4336 writesame 0xec08681 thru 0xec0a851 data_size 0x1(0x21d1 bytes) 2024-09-22T01:02:13.765 INFO:teuthology.orchestra.run.smithi002.stdout:4338 read 0x85899b6 thru 0x8593d6b (0xa3b6 bytes) 2024-09-22T01:02:13.765 INFO:teuthology.orchestra.run.smithi002.stdout:4339 write 0x47d35e2 thru 0x47dfa4f (0xc46e bytes) 2024-09-22T01:02:13.765 INFO:teuthology.orchestra.run.smithi002.stdout:4341 read 0xd731925 thru 0xd73b6da (0x9db6 bytes) 2024-09-22T01:02:13.765 INFO:teuthology.orchestra.run.smithi002.stdout:4342 write 0x5cb98a9 thru 0x5cbf1c7 (0x591f bytes) 2024-09-22T01:02:13.766 INFO:teuthology.orchestra.run.smithi002.stdout:4343 writesame 0xbba6548 thru 0xbbacc4a data_size 0x1(0x6703 bytes) 2024-09-22T01:02:13.766 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:13.762+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:13.794 INFO:teuthology.orchestra.run.smithi002.stdout:4344 flatten 2024-09-22T01:02:13.837 INFO:teuthology.orchestra.run.smithi002.stdout:4345 read 0x69ea68 thru 0x6a7eab (0x9444 bytes) 2024-09-22T01:02:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:4346 punch from 0x61507d0 to 0x615b053, (0xa883 bytes) 2024-09-22T01:02:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:4347 read 0xcef7e66 thru 0xcf0794c (0xfae7 bytes) 2024-09-22T01:02:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:4351 write 0xc36ff52 thru 0xc37fe0c (0xfebb bytes) 2024-09-22T01:02:13.843 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:02:13.843 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T01:02:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:4352 write 0xde935cc thru 0xdea0eac (0xd8e1 bytes) 2024-09-22T01:02:13.846 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:02:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:4354 write 0x26fcb7c thru 0x27023dd (0x5862 bytes) 2024-09-22T01:02:13.863 INFO:teuthology.orchestra.run.smithi002.stdout:4355 writesame 0xc3b20b3 thru 0xc3b352b data_size 0x1(0x1479 bytes) 2024-09-22T01:02:13.864 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:13.859+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:13.870 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:13.892 INFO:teuthology.orchestra.run.smithi002.stdout:4358 trunc from 0xec0a852 to 0xcf8025 2024-09-22T01:02:13.908 INFO:teuthology.orchestra.run.smithi002.stdout:4359 read 0x278035 thru 0x28342e (0xb3fa bytes) 2024-09-22T01:02:13.932 INFO:teuthology.orchestra.run.smithi002.stdout:4360 write 0x9bee235 thru 0x9befbf3 (0x19bf bytes) 2024-09-22T01:02:13.937 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:13.947 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:13.947 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:13.947 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:13.947 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:13.949 INFO:teuthology.orchestra.run.smithi002.stdout:4363 writesame 0xccd7b28 thru 0xccd7e77 data_size 0x8(0x350 bytes) 2024-09-22T01:02:13.949 INFO:teuthology.orchestra.run.smithi002.stdout:4364 read 0x38866a0 thru 0x388ff32 (0x9893 bytes) 2024-09-22T01:02:13.949 INFO:teuthology.orchestra.run.smithi002.stdout:4365 read 0xafc48e1 thru 0xafccb76 (0x8296 bytes) 2024-09-22T01:02:13.949 INFO:teuthology.orchestra.run.smithi002.stdout:4367 trunc from 0xccd7e78 to 0xc7308ce 2024-09-22T01:02:13.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:13.965 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:13.962+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:13.969 INFO:teuthology.orchestra.run.smithi002.stdout:4368 trunc from 0xc7308ce to 0x7145a02 2024-09-22T01:02:13.986 INFO:teuthology.orchestra.run.smithi002.stdout:4369 write 0x3c31cf3 thru 0x3c37271 (0x557f bytes) 2024-09-22T01:02:13.990 INFO:teuthology.orchestra.run.smithi002.stdout:4370 read 0x30379b6 thru 0x3040a12 (0x905d bytes) 2024-09-22T01:02:13.990 INFO:teuthology.orchestra.run.smithi002.stdout:4371 punch from 0x3ec4e3e to 0x3ed3fe0, (0xf1a2 bytes) 2024-09-22T01:02:13.991 INFO:teuthology.orchestra.run.smithi002.stdout:4373 punch from 0x49fde95 to 0x4a00cb9, (0x2e24 bytes) 2024-09-22T01:02:14.002 INFO:teuthology.orchestra.run.smithi002.stdout:4374 write 0x8620881 thru 0x862e875 (0xdff5 bytes) 2024-09-22T01:02:14.019 INFO:teuthology.orchestra.run.smithi002.stdout:4375 write 0xc4638f5 thru 0xc46bcfa (0x8406 bytes) 2024-09-22T01:02:14.023 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:14.023 INFO:teuthology.orchestra.run.smithi002.stdout:4377 writesame 0x24bb377 thru 0x24c4aaf data_size 0x1(0x9739 bytes) 2024-09-22T01:02:14.056 INFO:teuthology.orchestra.run.smithi002.stdout:4378 write 0x360c1f6 thru 0x3619025 (0xce30 bytes) 2024-09-22T01:02:14.066 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:14.066+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:14.071 INFO:teuthology.orchestra.run.smithi002.stdout:4380 write 0xd9810e8 thru 0xd98dc56 (0xcb6f bytes) 2024-09-22T01:02:14.076 INFO:teuthology.orchestra.run.smithi002.stdout:4381 write 0xabde6e3 thru 0xabdfd55 (0x1673 bytes) 2024-09-22T01:02:14.081 INFO:teuthology.orchestra.run.smithi002.stdout:4382 write 0x759d09d thru 0x75ab715 (0xe679 bytes) 2024-09-22T01:02:14.087 INFO:teuthology.orchestra.run.smithi002.stdout:4384 punch from 0x3035ad6 to 0x303d6ea, (0x7c14 bytes) 2024-09-22T01:02:14.087 INFO:teuthology.orchestra.run.smithi002.stdout:4385 write 0x29d606d thru 0x29e138c (0xb320 bytes) 2024-09-22T01:02:14.092 INFO:teuthology.orchestra.run.smithi002.stdout:4387 writesame 0x3a65156 thru 0x3a713c6 data_size 0xd(0xc271 bytes) 2024-09-22T01:02:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:4388 trunc from 0xd98dc57 to 0x416c5a7 2024-09-22T01:02:14.125 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:14.126 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:14.140 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:14.159 INFO:teuthology.orchestra.run.smithi002.stdout:4389 write 0x7f6d1fb thru 0x7f735fe (0x6404 bytes) 2024-09-22T01:02:14.180 INFO:teuthology.orchestra.run.smithi002.stdout:4390 write 0x9b28e43 thru 0x9b36e6c (0xe02a bytes) 2024-09-22T01:02:14.184 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:14.183+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:14.186 INFO:teuthology.orchestra.run.smithi002.stdout:4391 read 0x6388e26 thru 0x6398775 (0xf950 bytes) 2024-09-22T01:02:14.186 INFO:teuthology.orchestra.run.smithi002.stdout:4392 read 0x98c7350 thru 0x98c8e11 (0x1ac2 bytes) 2024-09-22T01:02:14.186 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:14.186 INFO:teuthology.orchestra.run.smithi002.stdout:4394 write 0x647c4c6 thru 0x647d0d2 (0xc0d bytes) 2024-09-22T01:02:14.192 INFO:teuthology.orchestra.run.smithi002.stdout:4395 read 0x1d3baee thru 0x1d46ec7 (0xb3da bytes) 2024-09-22T01:02:14.193 INFO:teuthology.orchestra.run.smithi002.stdout:4396 write 0x6b74419 thru 0x6b810ec (0xccd4 bytes) 2024-09-22T01:02:14.198 INFO:teuthology.orchestra.run.smithi002.stdout:4397 read 0x3d4d150 thru 0x3d5bbbb (0xea6c bytes) 2024-09-22T01:02:14.198 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:14.198 INFO:teuthology.orchestra.run.smithi002.stdout:4401 punch from 0x8685bb1 to 0x8686652, (0xaa1 bytes) 2024-09-22T01:02:14.200 INFO:teuthology.orchestra.run.smithi002.stdout:4402 trunc from 0x9b36e6d to 0xb914f47 2024-09-22T01:02:14.213 INFO:teuthology.orchestra.run.smithi002.stdout:4403 writesame 0x560c64c thru 0x5619ab7 data_size 0xa(0xd46c bytes) 2024-09-22T01:02:14.220 INFO:teuthology.orchestra.run.smithi002.stdout:4405 writesame 0x2dff041 thru 0x2e0e09e data_size 0xb(0xf05e bytes) 2024-09-22T01:02:14.227 INFO:teuthology.orchestra.run.smithi002.stdout:4406 punch from 0x41111b4 to 0x411fd97, (0xebe3 bytes) 2024-09-22T01:02:14.227 INFO:teuthology.orchestra.run.smithi002.stdout:4407 write 0xaf2b078 thru 0xaf36b6a (0xbaf3 bytes) 2024-09-22T01:02:14.233 INFO:teuthology.orchestra.run.smithi002.stdout:4408 read 0x352a3a4 thru 0x3537ca7 (0xd904 bytes) 2024-09-22T01:02:14.233 INFO:teuthology.orchestra.run.smithi002.stdout:4409 read 0x11c0eb3 thru 0x11d0624 (0xf772 bytes) 2024-09-22T01:02:14.235 INFO:teuthology.orchestra.run.smithi002.stdout:4410 read 0x9d01b0d thru 0x9d047ac (0x2ca0 bytes) 2024-09-22T01:02:14.235 INFO:teuthology.orchestra.run.smithi002.stdout:4411 read 0x79641b2 thru 0x796d646 (0x9495 bytes) 2024-09-22T01:02:14.251 INFO:teuthology.orchestra.run.smithi002.stdout:4412 write 0xd28c016 thru 0xd2965bd (0xa5a8 bytes) 2024-09-22T01:02:14.258 INFO:teuthology.orchestra.run.smithi002.stdout:4413 punch from 0x98fd392 to 0x9908230, (0xae9e bytes) 2024-09-22T01:02:14.258 INFO:teuthology.orchestra.run.smithi002.stdout:4416 read 0x3233097 thru 0x3238ca1 (0x5c0b bytes) 2024-09-22T01:02:14.258 INFO:teuthology.orchestra.run.smithi002.stdout:4417 write 0x486d823 thru 0x486e30c (0xaea bytes) 2024-09-22T01:02:14.263 INFO:teuthology.orchestra.run.smithi002.stdout:4418 read 0x13e88e6 thru 0x13eebf1 (0x630c bytes) 2024-09-22T01:02:14.267 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:14.263+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912635362, 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:02:14.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:02:14.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912598393, 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.276 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.280 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912570123999998, 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:02:14.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:02:14.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912532248000002, 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912493657999999, 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:14.293 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912421330000001, 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912383464000001, 2024-09-22T01:02:14.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:02:14.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:02:14.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912341224999999, 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:14.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:14.306 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.91228684, 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912186075000001, 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:02:14.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.91209915, 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:14.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:02:14.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:02:14.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:14.319 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.912040825999998, 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.911587676, 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:02:14.327 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.91094919, 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:02:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:02:14.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:02:14.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.910661593, 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:02:14.332 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:14.336 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.910058624000001, 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:02:14.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.908978325, 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:02:14.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:02:14.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.908908447000002, 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:02:14.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:02:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.908346456, 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.349 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:02:14.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.907969031, 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.353 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:02:14.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.357 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:14.358 INFO:teuthology.orchestra.run.smithi002.stdout:4419 write 0xd81dabf thru 0xd82154e (0x3a90 bytes) 2024-09-22T01:02:14.359 INFO:teuthology.orchestra.run.smithi002.stdout:4420 read 0x9f9a111 thru 0x9f9f389 (0x5279 bytes) 2024-09-22T01:02:14.359 INFO:teuthology.orchestra.run.smithi002.stdout:4421 write 0x82ae794 thru 0x82b0bd0 (0x243d bytes) 2024-09-22T01:02:14.359 INFO:teuthology.orchestra.run.smithi002.stdout:4423 clone 38 order 24 su 65536 sc 2 2024-09-22T01:02:14.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:14.369 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:14.366+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:14.470 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:14.470+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:14.527 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:14.527 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:14.527 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:14.527 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:14.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:14.571 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:14.570+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:14.659 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:14.659 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:14.659 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:14.659 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:14.659 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:14.672 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:14.670+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:14.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:14.772 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:14.767+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:14.792 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:14.792 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 102.899744899, 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.793 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:02:14.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:02:14.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.799 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515747572000002, 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:02:14.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515714267, 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:02:14.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.807 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515617753999997, 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.808 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:02:14.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:02:14.811 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515544503000001, 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.815 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515479321999997, 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.816 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515409200999997, 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.820 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:02:14.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515375835999997, 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:02:14.824 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:02:14.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:02:14.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515345414000002, 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:02:14.828 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:02:14.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:02:14.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515315108000003, 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:02:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:02:14.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:02:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515284706000003, 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:02:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:02:14.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:02:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515253907000002, 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515223433999999, 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:02:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:02:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515192841999998, 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:02:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:02:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515155040000003, 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:02:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:02:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515122675000001, 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:02:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515092355999997, 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515062464000003, 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:02:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:02:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.515032705000003, 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:02:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.514982240000002, 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:14.878 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:14.878 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T01:02:14.879 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:14.880 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:14.878+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:14.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T01:02:14.980 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:14.978+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:15.056 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:15.056 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:15.056 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:15.056 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:15.069 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:15.112 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:15.111+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:15.143 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-22T01:02:15.181 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:15.179+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:15.197 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:15.198 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:15.198 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:15.198 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:15.198 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:15.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:15.235 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:15.300 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:15.299+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:15.383 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:15.379+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252974293999998, 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252918618999999, 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.390 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:02:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252866273999999, 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:02:15.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:02:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252808363, 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:02:15.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:02:15.405 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252749723999997, 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.406 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252694492000003, 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:02:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:02:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252640767999999, 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:02:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:02:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252569033, 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:02:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:02:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252511652000003, 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252461386, 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252382329, 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252258079999997, 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:02:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:02:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:02:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:02:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252177498000002, 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.252112310000001, 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:02:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:02:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:02:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.251337341000003, 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.251287556000001, 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:02:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.251206455000002, 2024-09-22T01:02:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:02:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.251136469999999, 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:02:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:02:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.251059106, 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:02:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:02:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.250998584999998, 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:02:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:02:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:15.488 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:15.488 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:15.489 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:15.489 INFO:teuthology.orchestra.run.smithi002.stdout:4425 trunc from 0xd82154f to 0x277e9cc 2024-09-22T01:02:15.490 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:15.490 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:15.490 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:15.487+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:15.491 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:15.491 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:15.491 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:15.492 INFO:teuthology.orchestra.run.smithi002.stdout:4426 read 0x1f4bbc9 thru 0x1f54abb (0x8ef3 bytes) 2024-09-22T01:02:15.492 INFO:teuthology.orchestra.run.smithi002.stdout:4427 read 0x747ce6 thru 0x74ed4c (0x7067 bytes) 2024-09-22T01:02:15.492 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:15.492 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:15.493 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:15.493 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:15.497 INFO:teuthology.orchestra.run.smithi002.stdout:4428 write 0xb5eae1f thru 0xb5f41ef (0x93d1 bytes) 2024-09-22T01:02:15.505 INFO:teuthology.orchestra.run.smithi002.stdout:4429 read 0xa9ba585 thru 0xa9c5f9c (0xba18 bytes) 2024-09-22T01:02:15.505 INFO:teuthology.orchestra.run.smithi002.stdout:4430 writesame 0x209a933 thru 0x20a5f5a data_size 0x1d(0xb628 bytes) 2024-09-22T01:02:15.511 INFO:teuthology.orchestra.run.smithi002.stdout:4431 read 0x805a211 thru 0x8063787 (0x9577 bytes) 2024-09-22T01:02:15.511 INFO:teuthology.orchestra.run.smithi002.stdout:4432 writesame 0x3bc13dd thru 0x3bc1b1e data_size 0x1(0x742 bytes) 2024-09-22T01:02:15.515 INFO:teuthology.orchestra.run.smithi002.stdout:4433 punch from 0x80ecfd7 to 0x80ef1ab, (0x21d4 bytes) 2024-09-22T01:02:15.515 INFO:teuthology.orchestra.run.smithi002.stdout:4435 punch from 0x7fde133 to 0x7fe8556, (0xa423 bytes) 2024-09-22T01:02:15.515 INFO:teuthology.orchestra.run.smithi002.stdout:4436 read 0x4bce152 thru 0x4bd1b63 (0x3a12 bytes) 2024-09-22T01:02:15.515 INFO:teuthology.orchestra.run.smithi002.stdout:4439 read 0x46bb225 thru 0x46c30ab (0x7e87 bytes) 2024-09-22T01:02:15.517 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:15.529 INFO:teuthology.orchestra.run.smithi002.stdout:4442 write 0xcd48dd8 thru 0xcd4d89e (0x4ac7 bytes) 2024-09-22T01:02:15.535 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:15.535 INFO:teuthology.orchestra.run.smithi002.stdout:4447 write 0xb56ec00 thru 0xb575917 (0x6d18 bytes) 2024-09-22T01:02:15.538 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:15.539 INFO:teuthology.orchestra.run.smithi002.stdout:4448 writesame 0x7a84749 thru 0x7a86062 data_size 0x36(0x191a bytes) 2024-09-22T01:02:15.546 INFO:teuthology.orchestra.run.smithi002.stdout:4451 read 0x4840981 thru 0x484c835 (0xbeb5 bytes) 2024-09-22T01:02:15.546 INFO:teuthology.orchestra.run.smithi002.stdout:4452 punch from 0x87e8d63 to 0x87e9e6f, (0x110c bytes) 2024-09-22T01:02:15.552 INFO:teuthology.orchestra.run.smithi002.stdout:4455 trunc from 0xcd4d89f to 0x8fa5c27 2024-09-22T01:02:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:15.562 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:15.575 INFO:teuthology.orchestra.run.smithi002.stdout:4456 read 0x79df8c6 thru 0x79e8475 (0x8bb0 bytes) 2024-09-22T01:02:15.576 INFO:teuthology.orchestra.run.smithi002.stdout:4457 writesame 0x5fe4057 thru 0x5fef291 data_size 0x43(0xb23b bytes) 2024-09-22T01:02:15.583 INFO:teuthology.orchestra.run.smithi002.stdout:4458 write 0x696583c thru 0x696d604 (0x7dc9 bytes) 2024-09-22T01:02:15.586 INFO:teuthology.orchestra.run.smithi002.stdout:4459 write 0x3d07a4d thru 0x3d09cdc (0x2290 bytes) 2024-09-22T01:02:15.590 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:15.587+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:15.602 INFO:teuthology.orchestra.run.smithi002.stdout:4460 write 0x960e90b thru 0x960f357 (0xa4d bytes) 2024-09-22T01:02:15.608 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:15.608 INFO:teuthology.orchestra.run.smithi002.stdout:4462 punch from 0x422ddfb to 0x423c8b6, (0xeabb bytes) 2024-09-22T01:02:15.625 INFO:teuthology.orchestra.run.smithi002.stdout:4464 write 0xc44e2ad thru 0xc45478c (0x64e0 bytes) 2024-09-22T01:02:15.626 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:15.626 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:15.634 INFO:teuthology.orchestra.run.smithi002.stdout:4465 punch from 0x7b9ccef to 0x7b9dd61, (0x1072 bytes) 2024-09-22T01:02:15.634 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:15.634 INFO:teuthology.orchestra.run.smithi002.stdout:4468 write 0x2e7987e thru 0x2e7b7f9 (0x1f7c bytes) 2024-09-22T01:02:15.645 INFO:teuthology.orchestra.run.smithi002.stdout:4470 trunc from 0xc45478d to 0x790b2a0 2024-09-22T01:02:15.645 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:15.673 INFO:teuthology.orchestra.run.smithi002.stdout:4471 read 0x769c06d thru 0x769ebc2 (0x2b56 bytes) 2024-09-22T01:02:15.673 INFO:teuthology.orchestra.run.smithi002.stdout:4472 punch from 0x966ad5 to 0x96b73e, (0x4c69 bytes) 2024-09-22T01:02:15.673 INFO:teuthology.orchestra.run.smithi002.stdout:4474 read 0x11aa3f5 thru 0x11b9e7b (0xfa87 bytes) 2024-09-22T01:02:15.674 INFO:teuthology.orchestra.run.smithi002.stdout:4475 read 0x3db655a thru 0x3db9cb9 (0x3760 bytes) 2024-09-22T01:02:15.674 INFO:teuthology.orchestra.run.smithi002.stdout:4477 write 0x1578010 thru 0x157d2fa (0x52eb bytes) 2024-09-22T01:02:15.691 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:15.687+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:15.732 INFO:teuthology.orchestra.run.smithi002.stdout:4478 trunc from 0x790b2a0 to 0xb5f6ac0 2024-09-22T01:02:15.746 INFO:teuthology.orchestra.run.smithi002.stdout:4480 writesame 0x65705cb thru 0x657774f data_size 0x3(0x7185 bytes) 2024-09-22T01:02:15.777 INFO:teuthology.orchestra.run.smithi002.stdout:4481 writesame 0xbcba45c thru 0xbcc53d8 data_size 0x19(0xaf7d bytes) 2024-09-22T01:02:15.777 INFO:teuthology.orchestra.run.smithi002.stdout:4483 punch from 0x949bfa5 to 0x94a10be, (0x5119 bytes) 2024-09-22T01:02:15.777 INFO:teuthology.orchestra.run.smithi002.stdout:4484 writesame 0x3e64432 thru 0x3e7104c data_size 0x1(0xcc1b bytes) 2024-09-22T01:02:15.792 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:15.787+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:15.804 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:15.804 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:15.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:15.827 INFO:teuthology.orchestra.run.smithi002.stdout:4486 write 0xb042dc thru 0xb10c0a (0xc92f bytes) 2024-09-22T01:02:15.863 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T01:02:15.876 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:02:15.883 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T01:02:15.898 INFO:teuthology.orchestra.run.smithi002.stdout:4487 punch from 0x762933a to 0x762c616, (0x32dc bytes) 2024-09-22T01:02:15.898 INFO:teuthology.orchestra.run.smithi002.stdout:4488 punch from 0x7872e82 to 0x7879c2e, (0x6dac bytes) 2024-09-22T01:02:15.898 INFO:teuthology.orchestra.run.smithi002.stdout:4489 writesame 0x614b2ba thru 0x615a537 data_size 0x1(0xf27e bytes) 2024-09-22T01:02:15.898 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:15.898+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:15.947 INFO:teuthology.orchestra.run.smithi002.stdout:4491 write 0x1d5998a thru 0x1d5a4d5 (0xb4c bytes) 2024-09-22T01:02:15.950 INFO:teuthology.orchestra.run.smithi002.stdout:4492 read 0x244de40 thru 0x2453d12 (0x5ed3 bytes) 2024-09-22T01:02:15.955 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:15.955 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:15.969 INFO:teuthology.orchestra.run.smithi002.stdout:4496 write 0xd2bd1a9 thru 0xd2c2b04 (0x595c bytes) 2024-09-22T01:02:15.974 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:15.977 INFO:teuthology.orchestra.run.smithi002.stdout:4498 writesame 0x1635939 thru 0x163b77b data_size 0x1(0x5e43 bytes) 2024-09-22T01:02:15.991 INFO:teuthology.orchestra.run.smithi002.stdout:4502 write 0xcc6e2c0 thru 0xcc7df15 (0xfc56 bytes) 2024-09-22T01:02:15.994 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:15.991+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:15.995 INFO:teuthology.orchestra.run.smithi002.stdout:4503 read 0xb9e8078 thru 0xb9f68c0 (0xe849 bytes) 2024-09-22T01:02:15.995 INFO:teuthology.orchestra.run.smithi002.stdout:4504 read 0x99130f4 thru 0x99130fa (0x7 bytes) 2024-09-22T01:02:15.996 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:15.996 INFO:teuthology.orchestra.run.smithi002.stdout:4508 writesame 0xc8da3e0 thru 0xc8e1e26 data_size 0x1(0x7a47 bytes) 2024-09-22T01:02:16.012 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.012 INFO:teuthology.orchestra.run.smithi002.stdout:4510 write 0xaca37e4 thru 0xaca395d (0x17a bytes) 2024-09-22T01:02:16.020 INFO:teuthology.orchestra.run.smithi002.stdout:4511 read 0x1d5433d thru 0x1d63272 (0xef36 bytes) 2024-09-22T01:02:16.020 INFO:teuthology.orchestra.run.smithi002.stdout:4512 read 0x288430a thru 0x288c62b (0x8322 bytes) 2024-09-22T01:02:16.021 INFO:teuthology.orchestra.run.smithi002.stdout:4514 write 0x234aaa4 thru 0x23598e7 (0xee44 bytes) 2024-09-22T01:02:16.038 INFO:teuthology.orchestra.run.smithi002.stdout:4515 write 0xe016132 thru 0xe019452 (0x3321 bytes) 2024-09-22T01:02:16.045 INFO:teuthology.orchestra.run.smithi002.stdout:4516 write 0x23035e6 thru 0x230e420 (0xae3b bytes) 2024-09-22T01:02:16.049 INFO:teuthology.orchestra.run.smithi002.stdout:4517 write 0x965c2c9 thru 0x9669f13 (0xdc4b bytes) 2024-09-22T01:02:16.058 INFO:teuthology.orchestra.run.smithi002.stdout:4519 read 0x234bdd9 thru 0x23589f8 (0xcc20 bytes) 2024-09-22T01:02:16.059 INFO:teuthology.orchestra.run.smithi002.stdout:4520 punch from 0xa607df to 0xa62e30, (0x2651 bytes) 2024-09-22T01:02:16.059 INFO:teuthology.orchestra.run.smithi002.stdout:4521 writesame 0xd236dfe thru 0xd2413cd data_size 0x38(0xa5d0 bytes) 2024-09-22T01:02:16.063 INFO:teuthology.orchestra.run.smithi002.stdout:4522 write 0xc349d43 thru 0xc35481a (0xaad8 bytes) 2024-09-22T01:02:16.066 INFO:teuthology.orchestra.run.smithi002.stdout:4523 write 0x914ec17 thru 0x91578fb (0x8ce5 bytes) 2024-09-22T01:02:16.069 INFO:teuthology.orchestra.run.smithi002.stdout:4524 punch from 0x1e6dc2c to 0x1e7a083, (0xc457 bytes) 2024-09-22T01:02:16.078 INFO:teuthology.orchestra.run.smithi002.stdout:4525 trunc from 0xe019453 to 0x7ab0b3e 2024-09-22T01:02:16.095 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:16.091+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:16.120 INFO:teuthology.orchestra.run.smithi002.stdout:4527 writesame 0x97a70b5 thru 0x97ac46a data_size 0x5(0x53b6 bytes) 2024-09-22T01:02:16.120 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.120 INFO:teuthology.orchestra.run.smithi002.stdout:4529 write 0x5a40fba thru 0x5a4c885 (0xb8cc bytes) 2024-09-22T01:02:16.123 INFO:teuthology.orchestra.run.smithi002.stdout:4530 punch from 0x44a1362 to 0x44acde2, (0xba80 bytes) 2024-09-22T01:02:16.123 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.123 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.123 INFO:teuthology.orchestra.run.smithi002.stdout:4538 punch from 0x33f84a7 to 0x33fb749, (0x32a2 bytes) 2024-09-22T01:02:16.123 INFO:teuthology.orchestra.run.smithi002.stdout:4539 punch from 0x86cd19e to 0x86d616b, (0x8fcd bytes) 2024-09-22T01:02:16.123 INFO:teuthology.orchestra.run.smithi002.stdout:4541 punch from 0x44e5d15 to 0x44f1351, (0xb63c bytes) 2024-09-22T01:02:16.124 INFO:teuthology.orchestra.run.smithi002.stdout:4542 punch from 0x4793efd to 0x47a0f40, (0xd043 bytes) 2024-09-22T01:02:16.124 INFO:teuthology.orchestra.run.smithi002.stdout:4543 writesame 0x405eb5c thru 0x405f007 data_size 0x17(0x4ac bytes) 2024-09-22T01:02:16.130 INFO:teuthology.orchestra.run.smithi002.stdout:4544 punch from 0x774ab7e to 0x7759598, (0xea1a bytes) 2024-09-22T01:02:16.139 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:16.139 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:16.150 INFO:teuthology.orchestra.run.smithi002.stdout:4545 writesame 0xd1064e6 thru 0xd1115cd data_size 0x66(0xb0e8 bytes) 2024-09-22T01:02:16.155 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:16.166 INFO:teuthology.orchestra.run.smithi002.stdout:4546 write 0xe9e4025 thru 0xe9f1c07 (0xdbe3 bytes) 2024-09-22T01:02:16.175 INFO:teuthology.orchestra.run.smithi002.stdout:4547 writesame 0xda48203 thru 0xda55d2e data_size 0x34(0xdb2c bytes) 2024-09-22T01:02:16.184 INFO:teuthology.orchestra.run.smithi002.stdout:4548 punch from 0x9e267b2 to 0x9e2e077, (0x78c5 bytes) 2024-09-22T01:02:16.184 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.196 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:16.194+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:16.203 INFO:teuthology.orchestra.run.smithi002.stdout:4552 trunc from 0xe9f1c08 to 0x2c3f494 2024-09-22T01:02:16.232 INFO:teuthology.orchestra.run.smithi002.stdout:4553 punch from 0x27a886b to 0x27b840c, (0xfba1 bytes) 2024-09-22T01:02:16.256 INFO:teuthology.orchestra.run.smithi002.stdout:4554 write 0x9d48942 thru 0x9d5748e (0xeb4d bytes) 2024-09-22T01:02:16.265 INFO:teuthology.orchestra.run.smithi002.stdout:4555 write 0x30bedda thru 0x30caab8 (0xbcdf bytes) 2024-09-22T01:02:16.269 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.269 INFO:teuthology.orchestra.run.smithi002.stdout:4559 writesame 0x1a53390 thru 0x1a58c8d data_size 0x3(0x58fe bytes) 2024-09-22T01:02:16.290 INFO:teuthology.orchestra.run.smithi002.stdout:4560 write 0xcfd982c thru 0xcfdcfcb (0x37a0 bytes) 2024-09-22T01:02:16.295 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:16.295 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:16.296 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.296 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:16.291+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:16.297 INFO:teuthology.orchestra.run.smithi002.stdout:4562 writesame 0x3f6a596 thru 0x3f78fe6 data_size 0x1(0xea51 bytes) 2024-09-22T01:02:16.309 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:16.351 INFO:teuthology.orchestra.run.smithi002.stdout:4563 read 0x8bf6a8c thru 0x8bff620 (0x8b95 bytes) 2024-09-22T01:02:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:4564 writesame 0x36b30ff thru 0x36c0158 data_size 0x1(0xd05a bytes) 2024-09-22T01:02:16.396 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:16.395+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:16.409 INFO:teuthology.orchestra.run.smithi002.stdout:4565 read 0x3888a64 thru 0x3891072 (0x860f bytes) 2024-09-22T01:02:16.410 INFO:teuthology.orchestra.run.smithi002.stdout:4567 punch from 0xaca354 to 0xad52a4, (0xaf50 bytes) 2024-09-22T01:02:16.410 INFO:teuthology.orchestra.run.smithi002.stdout:4570 punch from 0x850c0c8 to 0x851302f, (0x6f67 bytes) 2024-09-22T01:02:16.427 INFO:teuthology.orchestra.run.smithi002.stdout:4571 trunc from 0xcfdcfcc to 0x2a742da 2024-09-22T01:02:16.456 INFO:teuthology.orchestra.run.smithi002.stdout:4572 punch from 0x2797149 to 0x279a197, (0x304e bytes) 2024-09-22T01:02:16.456 INFO:teuthology.orchestra.run.smithi002.stdout:4574 read 0x1b709e5 thru 0x1b7c989 (0xbfa5 bytes) 2024-09-22T01:02:16.457 INFO:teuthology.orchestra.run.smithi002.stdout:4575 punch from 0x1bb44bf to 0x1bbd890, (0x93d1 bytes) 2024-09-22T01:02:16.457 INFO:teuthology.orchestra.run.smithi002.stdout:4576 write 0x242a686 thru 0x24344d3 (0x9e4e bytes) 2024-09-22T01:02:16.468 INFO:teuthology.orchestra.run.smithi002.stdout:4577 trunc from 0x2a742da to 0x64f03fb 2024-09-22T01:02:16.481 INFO:teuthology.orchestra.run.smithi002.stdout:4579 read 0x2969d74 thru 0x296d8c2 (0x3b4f bytes) 2024-09-22T01:02:16.481 INFO:teuthology.orchestra.run.smithi002.stdout:4580 write 0x5fdb053 thru 0x5fe7072 (0xc020 bytes) 2024-09-22T01:02:16.484 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:16.484 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:16.492 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.493 INFO:teuthology.orchestra.run.smithi002.stdout:4582 punch from 0x8a80ee to 0x8ac566, (0x4478 bytes) 2024-09-22T01:02:16.493 INFO:teuthology.orchestra.run.smithi002.stdout:4584 read 0x3a8c2b2 thru 0x3a91f24 (0x5c73 bytes) 2024-09-22T01:02:16.497 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:16.494+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:16.500 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:16.524 INFO:teuthology.orchestra.run.smithi002.stdout:4585 writesame 0xebfb565 thru 0xec0087a data_size 0xf(0x5316 bytes) 2024-09-22T01:02:16.524 INFO:teuthology.orchestra.run.smithi002.stdout:4587 write 0x65041bd thru 0x650c13c (0x7f80 bytes) 2024-09-22T01:02:16.531 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.531 INFO:teuthology.orchestra.run.smithi002.stdout:4589 write 0x6865855 thru 0x687375c (0xdf08 bytes) 2024-09-22T01:02:16.539 INFO:teuthology.orchestra.run.smithi002.stdout:4590 write 0x87cfaa8 thru 0x87d90ae (0x9607 bytes) 2024-09-22T01:02:16.549 INFO:teuthology.orchestra.run.smithi002.stdout:4591 read 0x6d4f446 thru 0x6d51442 (0x1ffd bytes) 2024-09-22T01:02:16.574 INFO:teuthology.orchestra.run.smithi002.stdout:4593 trunc from 0xec0087b to 0xd426b6 2024-09-22T01:02:16.598 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:16.598+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:16.602 INFO:teuthology.orchestra.run.smithi002.stdout:4594 punch from 0x7112b5 to 0x717d5f, (0x6aaa bytes) 2024-09-22T01:02:16.602 INFO:teuthology.orchestra.run.smithi002.stdout:4595 punch from 0xcd5d6f to 0xce3e12, (0xe0a3 bytes) 2024-09-22T01:02:16.622 INFO:teuthology.orchestra.run.smithi002.stdout:4596 writesame 0x54c2e2a thru 0x54c5e65 data_size 0x31(0x303c bytes) 2024-09-22T01:02:16.633 INFO:teuthology.orchestra.run.smithi002.stdout:4597 trunc from 0x54c5e66 to 0xc8afc98 2024-09-22T01:02:16.646 INFO:teuthology.orchestra.run.smithi002.stdout:4598 write 0x3b29c16 thru 0x3b30016 (0x6401 bytes) 2024-09-22T01:02:16.656 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.656 INFO:teuthology.orchestra.run.smithi002.stdout:4600 write 0x963c91b thru 0x963dca8 (0x138e bytes) 2024-09-22T01:02:16.660 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:16.660 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:16.663 INFO:teuthology.orchestra.run.smithi002.stdout:4601 write 0x856df46 thru 0x857ccfa (0xedb5 bytes) 2024-09-22T01:02:16.670 INFO:teuthology.orchestra.run.smithi002.stdout:4602 read 0xa01d84c thru 0xa02d2ec (0xfaa1 bytes) 2024-09-22T01:02:16.674 INFO:teuthology.orchestra.run.smithi002.stdout:4603 trunc from 0xc8afc98 to 0xe4b5519 2024-09-22T01:02:16.676 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:16.687 INFO:teuthology.orchestra.run.smithi002.stdout:4604 writesame 0x991be58 thru 0x99250f8 data_size 0x1(0x92a1 bytes) 2024-09-22T01:02:16.706 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.706 INFO:teuthology.orchestra.run.smithi002.stdout:4607 read 0x1e24430 thru 0x1e25b5d (0x172e bytes) 2024-09-22T01:02:16.706 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:16.706+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:16.707 INFO:teuthology.orchestra.run.smithi002.stdout:4608 read 0x11d243b thru 0x11d8ab1 (0x6677 bytes) 2024-09-22T01:02:16.707 INFO:teuthology.orchestra.run.smithi002.stdout:4610 punch from 0xc09b0f8 to 0xc0a7adf, (0xc9e7 bytes) 2024-09-22T01:02:16.707 INFO:teuthology.orchestra.run.smithi002.stdout:4611 read 0x7406376 thru 0x740cb33 (0x67be bytes) 2024-09-22T01:02:16.707 INFO:teuthology.orchestra.run.smithi002.stdout:4612 write 0xb630ceb thru 0xb63d76c (0xca82 bytes) 2024-09-22T01:02:16.716 INFO:teuthology.orchestra.run.smithi002.stdout:4613 read 0x814785 thru 0x818639 (0x3eb5 bytes) 2024-09-22T01:02:16.716 INFO:teuthology.orchestra.run.smithi002.stdout:4614 punch from 0x41b52ce to 0x41b9583, (0x42b5 bytes) 2024-09-22T01:02:16.738 INFO:teuthology.orchestra.run.smithi002.stdout:4615 trunc from 0xe4b5519 to 0x10ca26d 2024-09-22T01:02:16.765 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.765 INFO:teuthology.orchestra.run.smithi002.stdout:4617 read 0xcf23d6 thru 0xcfade8 (0x8a13 bytes) 2024-09-22T01:02:16.766 INFO:teuthology.orchestra.run.smithi002.stdout:4618 punch from 0x345e9c to 0x34befa, (0x605e bytes) 2024-09-22T01:02:16.766 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:16.787 INFO:teuthology.orchestra.run.smithi002.stdout:4621 write 0x685d93d thru 0x68615c8 (0x3c8c bytes) 2024-09-22T01:02:16.797 INFO:teuthology.orchestra.run.smithi002.stdout:4622 write 0x4c5404a thru 0x4c56f8a (0x2f41 bytes) 2024-09-22T01:02:16.799 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:16.794+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:16.810 INFO:teuthology.orchestra.run.smithi002.stdout:4623 trunc from 0x68615c9 to 0xa2bbf86 2024-09-22T01:02:16.814 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:16.814 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:16.823 INFO:teuthology.orchestra.run.smithi002.stdout:4624 write 0x3a4b0b3 thru 0x3a53f16 (0x8e64 bytes) 2024-09-22T01:02:16.829 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:16.834 INFO:teuthology.orchestra.run.smithi002.stdout:4625 write 0x9b50ea1 thru 0x9b5cf56 (0xc0b6 bytes) 2024-09-22T01:02:16.841 INFO:teuthology.orchestra.run.smithi002.stdout:4626 punch from 0x5778f73 to 0x5785f9b, (0xd028 bytes) 2024-09-22T01:02:16.841 INFO:teuthology.orchestra.run.smithi002.stdout:4627 read 0x58bca2b thru 0x58c63dc (0x99b2 bytes) 2024-09-22T01:02:16.843 INFO:teuthology.orchestra.run.smithi002.stdout:4628 trunc from 0xa2bbf86 to 0x98b932b 2024-09-22T01:02:16.859 INFO:teuthology.orchestra.run.smithi002.stdout:4629 clone 39 order 21 su 1048576 sc 2 2024-09-22T01:02:16.875 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:02:16.875 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T01:02:16.900 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:16.894+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:16.903 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T01:02:16.970 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:16.970 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:16.987 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:16.992 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone38 intact 2024-09-22T01:02:17.038 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:17.038+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:17.101 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:17.098+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:17.114 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:17.114 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:17.131 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:17.181 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-22T01:02:17.202 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:17.202+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:17.296 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:17.296 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:17.302 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:17.298+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:17.312 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:17.403 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:17.402+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:17.475 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:17.475 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:17.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:17.536 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:17.534+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:17.552 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:17.552 INFO:teuthology.orchestra.run.smithi002.stdout:4631 writesame 0x6e4b450 thru 0x6e57b94 data_size 0x1(0xc745 bytes) 2024-09-22T01:02:17.582 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:17.598 INFO:teuthology.orchestra.run.smithi002.stdout:4632 write 0xb48350e thru 0xb48cc2b (0x971e bytes) 2024-09-22T01:02:17.604 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:17.605 INFO:teuthology.orchestra.run.smithi002.stdout:4634 write 0x590d3e1 thru 0x5911b9f (0x47bf bytes) 2024-09-22T01:02:17.605 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:17.602+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:17.609 INFO:teuthology.orchestra.run.smithi002.stdout:4635 read 0x94ca631 thru 0x94d5a35 (0xb405 bytes) 2024-09-22T01:02:17.610 INFO:teuthology.orchestra.run.smithi002.stdout:4638 read 0x25c2795 thru 0x25d0aa2 (0xe30e bytes) 2024-09-22T01:02:17.611 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:17.621 INFO:teuthology.orchestra.run.smithi002.stdout:4641 write 0xe6c4469 thru 0xe6d32df (0xee77 bytes) 2024-09-22T01:02:17.624 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:17.624 INFO:teuthology.orchestra.run.smithi002.stdout:4644 write 0x6e09343 thru 0x6e100ad (0x6d6b bytes) 2024-09-22T01:02:17.626 INFO:teuthology.orchestra.run.smithi002.stdout:4645 write 0x862a261 thru 0x862a7ca (0x56a bytes) 2024-09-22T01:02:17.626 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:17.626 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:17.631 INFO:teuthology.orchestra.run.smithi002.stdout:4646 clone 40 order 18 su 16384 sc 8 2024-09-22T01:02:17.641 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:17.705 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:17.706+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:17.728 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:17.745 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:17.745 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:17.762 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:17.762 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:17.779 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:17.779 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:17.801 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:17.802 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:17.807 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:17.806+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:17.812 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:17.812 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:17.812 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:17.812 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:17.812 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:17.823 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:17.823 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:17.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:17.842 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:17.843 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:17.861 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:17.861 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:17.876 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T01:02:17.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-22T01:02:17.907 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:17.902+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985730062000002, 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:02:17.957 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:02:17.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:02:17.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985677539000001, 2024-09-22T01:02:17.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:17.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:02:17.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985625218000003, 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:02:17.965 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985459444, 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:17.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985431016, 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985403206999997, 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.977 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:02:17.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:02:17.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985375101999999, 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.984 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985347560000001, 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:02:17.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985319912000001, 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:02:17.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:02:17.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985292766999997, 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.993 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:02:17.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:17.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985264807999997, 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:17.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:02:17.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:02:17.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985236993000001, 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.001 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:02:18.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:02:18.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985210047999999, 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.008 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985182346999999, 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:18.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.012 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985067923000003, 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:02:18.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:02:18.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.016 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.985028565, 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:18.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:02:18.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.984998410000003, 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:02:18.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.024 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.984969151999998, 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.028 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.984941276000001, 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:18.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.984911902, 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:02:18.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:02:18.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.036 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:18.037 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone39 from 0xe6d32e0 (overlap 0x98b932b) to 0x89737a6 2024-09-22T01:02:18.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:18.038 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:18.034+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:18.138 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:18.134+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:18.204 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-22T01:02:18.214 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:18.214 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:18.214 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:18.214 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:18.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:18.273 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:18.270+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:18.340 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:18.338+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:18.391 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:18.392 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:18.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:18.449 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:18.446+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:18.505 INFO:teuthology.orchestra.run.smithi002.stdout:4648 write 0x6c8f5d9 thru 0x6c9af4c (0xb974 bytes) 2024-09-22T01:02:18.529 INFO:teuthology.orchestra.run.smithi002.stdout:4649 read 0x6ba8af4 thru 0x6baf678 (0x6b85 bytes) 2024-09-22T01:02:18.530 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:18.530 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:18.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:18.530 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170997444000001, 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:18.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:02:18.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170960475000001, 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:02:18.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:02:18.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170932206, 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:02:18.539 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170894329999999, 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.17085574, 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.548 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:02:18.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:02:18.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:02:18.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170783411999999, 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170745545999999, 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:18.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170703307, 2024-09-22T01:02:18.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:02:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:02:18.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.564 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170648922000002, 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170548156999999, 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.569 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:02:18.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:02:18.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:02:18.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170461232000001, 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:02:18.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.170402908, 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.581 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.169949758000001, 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.585 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.169311272000002, 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.586 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:02:18.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:02:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.169023674999998, 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:02:18.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:02:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.168420705999999, 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:02:18.594 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.167340407000001, 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.602 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.167270529, 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:02:18.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:02:18.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.166708538000002, 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.607 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:02:18.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:02:18.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.166331112999998, 2024-09-22T01:02:18.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.615 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:18.617 INFO:teuthology.orchestra.run.smithi002.stdout:4651 read 0x50c95b8 thru 0x50cd88f (0x42d8 bytes) 2024-09-22T01:02:18.617 INFO:teuthology.orchestra.run.smithi002.stdout:4652 punch from 0xa05b95e to 0xa067eba, (0xc55c bytes) 2024-09-22T01:02:18.617 INFO:teuthology.orchestra.run.smithi002.stdout:4655 trunc from 0xe6d32e0 to 0x6680f62 2024-09-22T01:02:18.617 INFO:teuthology.orchestra.run.smithi002.stdout:4657 write 0xda362d1 thru 0xda3bcc8 (0x59f8 bytes) 2024-09-22T01:02:18.618 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:18.618 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:18.614+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:18.630 INFO:teuthology.orchestra.run.smithi002.stdout:4659 trunc from 0xda3bcc9 to 0x7e84785 2024-09-22T01:02:18.654 INFO:teuthology.orchestra.run.smithi002.stdout:4660 write 0x32b76b2 thru 0x32b8525 (0xe74 bytes) 2024-09-22T01:02:18.678 INFO:teuthology.orchestra.run.smithi002.stdout:4661 read 0x4ccf319 thru 0x4cdb001 (0xbce9 bytes) 2024-09-22T01:02:18.680 INFO:teuthology.orchestra.run.smithi002.stdout:4664 read 0x6b4dbe0 thru 0x6b5aa0f (0xce30 bytes) 2024-09-22T01:02:18.680 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:18.706 INFO:teuthology.orchestra.run.smithi002.stdout:4667 write 0xea572f3 thru 0xea63cdb (0xc9e9 bytes) 2024-09-22T01:02:18.717 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:18.714+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:18.720 INFO:teuthology.orchestra.run.smithi002.stdout:4668 punch from 0x6ba54b2 to 0x6bb2e8d, (0xd9db bytes) 2024-09-22T01:02:18.720 INFO:teuthology.orchestra.run.smithi002.stdout:4669 writesame 0xc41265f thru 0xc4138ce data_size 0x2(0x1270 bytes) 2024-09-22T01:02:18.726 INFO:teuthology.orchestra.run.smithi002.stdout:4670 read 0xe18b21d thru 0xe1904c6 (0x52aa bytes) 2024-09-22T01:02:18.726 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:18.726 INFO:teuthology.orchestra.run.smithi002.stdout:4672 write 0xa2dc91c thru 0xa2e9a52 (0xd137 bytes) 2024-09-22T01:02:18.742 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:18.766 INFO:teuthology.orchestra.run.smithi002.stdout:4674 trunc from 0xea63cdc to 0x58a564d 2024-09-22T01:02:18.801 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:18.801 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:98584 3.f 3:f2446161:::rbd_data.117c612caf11.0000000000000162:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 262144 write_size 262144,truncate 81920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:02:18.800913+0000", 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00081102699999999997, 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00089324599999999999, 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "waiting for sub ops", 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 98584 2024-09-22T01:02:18.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.800913+0000", 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.800913+0000", 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.800916+0000", 2024-09-22T01:02:18.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8200000000000001e-06 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.800929+0000", 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3623e-05 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.800963+0000", 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3600999999999999e-05 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.800984+0000", 2024-09-22T01:02:18.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0588999999999999e-05 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.801068+0000", 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4029999999999993e-05 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.801201+0000", 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013323100000000001 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:02:18.801388+0000", 2024-09-22T01:02:18.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018706899999999999 2024-09-22T01:02:18.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.806 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:18.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:18.806 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-22T01:02:18.806 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 1 2024-09-22T01:02:18.806 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:18.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:18.819 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:18.818+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:18.848 INFO:teuthology.orchestra.run.smithi002.stdout:4675 write 0x9c92095 thru 0x9c9c47b (0xa3e7 bytes) 2024-09-22T01:02:18.856 INFO:teuthology.orchestra.run.smithi002.stdout:4676 write 0x93f8ca2 thru 0x9404879 (0xbbd8 bytes) 2024-09-22T01:02:18.876 INFO:teuthology.orchestra.run.smithi002.stdout:4677 trunc from 0x9c9c47c to 0xdcdadb1 2024-09-22T01:02:18.912 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:02:18.913 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.5 to 1 (10000) 2024-09-22T01:02:18.913 INFO:teuthology.orchestra.run.smithi002.stdout:4678 trunc from 0xdcdadb1 to 0x68b3bf3 2024-09-22T01:02:18.920 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-22T01:02:18.921 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:18.918+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:18.927 INFO:teuthology.orchestra.run.smithi002.stdout:4681 write 0x4050299 thru 0x40530aa (0x2e12 bytes) 2024-09-22T01:02:18.952 INFO:teuthology.orchestra.run.smithi002.stdout:4683 writesame 0x860df94 thru 0x861c37c data_size 0x7(0xe3e9 bytes) 2024-09-22T01:02:18.952 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:18.952 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:18.952 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:18.953 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:18.953 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:18.953 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:18.960 INFO:teuthology.orchestra.run.smithi002.stdout:4685 trunc from 0x861c37d to 0x3222a9a 2024-09-22T01:02:18.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:4686 writesame 0x62ea2a6 thru 0x62ec922 data_size 0x1(0x267d bytes) 2024-09-22T01:02:19.016 INFO:teuthology.orchestra.run.smithi002.stdout:4688 trunc from 0x62ec923 to 0x9fa97fc 2024-09-22T01:02:19.019 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:19.018+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:19.037 INFO:teuthology.orchestra.run.smithi002.stdout:4689 write 0xe286b5f thru 0xe28ad78 (0x421a bytes) 2024-09-22T01:02:19.044 INFO:teuthology.orchestra.run.smithi002.stdout:4690 write 0xabbcdd1 thru 0xabc400d (0x723d bytes) 2024-09-22T01:02:19.052 INFO:teuthology.orchestra.run.smithi002.stdout:4691 punch from 0xa406651 to 0xa4104fb, (0x9eaa bytes) 2024-09-22T01:02:19.052 INFO:teuthology.orchestra.run.smithi002.stdout:4692 punch from 0xada20ec to 0xada2223, (0x137 bytes) 2024-09-22T01:02:19.053 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:19.053 INFO:teuthology.orchestra.run.smithi002.stdout:4695 write 0x66b8c75 thru 0x66ba1a3 (0x152f bytes) 2024-09-22T01:02:19.056 INFO:teuthology.orchestra.run.smithi002.stdout:4696 punch from 0x57d2e8a to 0x57d6cd1, (0x3e47 bytes) 2024-09-22T01:02:19.056 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:19.056 INFO:teuthology.orchestra.run.smithi002.stdout:4698 write 0x7844004 thru 0x7848fef (0x4fec bytes) 2024-09-22T01:02:19.063 INFO:teuthology.orchestra.run.smithi002.stdout:4700 write 0xc21c04e thru 0xc222faf (0x6f62 bytes) 2024-09-22T01:02:19.067 INFO:teuthology.orchestra.run.smithi002.stdout:4702 trunc from 0xe28ad79 to 0xe0e670b 2024-09-22T01:02:19.080 INFO:teuthology.orchestra.run.smithi002.stdout:4703 punch from 0x3086143 to 0x3095624, (0xf4e1 bytes) 2024-09-22T01:02:19.082 INFO:teuthology.orchestra.run.smithi002.stdout:4704 trunc from 0xe0e670b to 0xcc5c69a 2024-09-22T01:02:19.089 INFO:teuthology.orchestra.run.smithi002.stdout:4705 read 0xb7a9dcd thru 0xb7b90b4 (0xf2e8 bytes) 2024-09-22T01:02:19.089 INFO:teuthology.orchestra.run.smithi002.stdout:4706 punch from 0x40452e8 to 0x404ae56, (0x5b6e bytes) 2024-09-22T01:02:19.089 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:19.089 INFO:teuthology.orchestra.run.smithi002.stdout:4708 write 0x20b4485 thru 0x20b9383 (0x4eff bytes) 2024-09-22T01:02:19.105 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:19.105 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 107.213897022, 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:02:19.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:02:19.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829899695000002, 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:02:19.112 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:02:19.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:02:19.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829866389999999, 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:02:19.116 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:02:19.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:02:19.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829769876999997, 2024-09-22T01:02:19.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:02:19.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:02:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829696626, 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:02:19.124 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829631444999997, 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.128 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:02:19.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:02:19.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829561323999997, 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:02:19.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829527959000004, 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829497537000002, 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829467231000002, 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.144 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:02:19.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829436829000002, 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.148 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:19.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829406030000001, 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.152 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829375556999999, 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829344964999997, 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:19.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829307163000003, 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829274798, 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.168 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:19.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829244479000003, 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.172 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:02:19.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829214587000003, 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:02:19.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829184828000002, 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.180 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:02:19.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.829134363000001, 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:02:19.184 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:19.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:02:19.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.188 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:19.190 INFO:teuthology.orchestra.run.smithi002.stdout:4709 write 0xcf22474 thru 0xcf2c4e6 (0xa073 bytes) 2024-09-22T01:02:19.190 INFO:teuthology.orchestra.run.smithi002.stdout:4710 trunc from 0xcf2c4e7 to 0x32f7299 2024-09-22T01:02:19.190 INFO:teuthology.orchestra.run.smithi002.stdout:4711 write 0x86618fb thru 0x866ad25 (0x942b bytes) 2024-09-22T01:02:19.190 INFO:teuthology.orchestra.run.smithi002.stdout:4712 punch from 0x72827e1 to 0x729256e, (0xfd8d bytes) 2024-09-22T01:02:19.190 INFO:teuthology.orchestra.run.smithi002.stdout:4713 write 0x23456e5 thru 0x2352a2d (0xd349 bytes) 2024-09-22T01:02:19.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:19.191 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:19.186+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:19.204 INFO:teuthology.orchestra.run.smithi002.stdout:4716 read 0x7e744fa thru 0x7e77239 (0x2d40 bytes) 2024-09-22T01:02:19.215 INFO:teuthology.orchestra.run.smithi002.stdout:4717 trunc from 0x866ad26 to 0x2be7056 2024-09-22T01:02:19.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T01:02:19.262+0000 7f4c6ffff640 0 --2- 172.21.15.2:0/1173336166 >> [v2:172.21.15.2:3301/0,v1:172.21.15.2:6790/0] conn(0x7f4c7813d060 0x7f4c78148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-22T01:02:19.263 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:19.281 INFO:teuthology.orchestra.run.smithi002.stdout:4719 write 0x89497de thru 0x894e248 (0x4a6b bytes) 2024-09-22T01:02:19.288 INFO:teuthology.orchestra.run.smithi002.stdout:4720 read 0x10974a5 thru 0x109fd37 (0x8893 bytes) 2024-09-22T01:02:19.290 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:19.290+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:19.311 INFO:teuthology.orchestra.run.smithi002.stdout:4721 writesame 0xe8fd4e2 thru 0xe90c238 data_size 0x81(0xed57 bytes) 2024-09-22T01:02:19.311 INFO:teuthology.orchestra.run.smithi002.stdout:4722 write 0xc2612be thru 0xc26632c (0x506f bytes) 2024-09-22T01:02:19.318 INFO:teuthology.orchestra.run.smithi002.stdout:4723 punch from 0x90d6dd5 to 0x90e2fb2, (0xc1dd bytes) 2024-09-22T01:02:19.330 INFO:teuthology.orchestra.run.smithi002.stdout:4724 trunc from 0xe90c239 to 0x958c1d2 2024-09-22T01:02:19.363 INFO:teuthology.orchestra.run.smithi002.stdout:4727 trunc from 0x958c1d2 to 0x2fea5af 2024-09-22T01:02:19.364 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:19.364 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:19.364 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:19.364 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:19.381 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:19.387 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:19.387 INFO:teuthology.orchestra.run.smithi002.stdout:4729 read 0xd3507d thru 0xd37d27 (0x2cab bytes) 2024-09-22T01:02:19.388 INFO:teuthology.orchestra.run.smithi002.stdout:4730 writesame 0x8ae84c thru 0x8b9aa1 data_size 0xe(0xb256 bytes) 2024-09-22T01:02:19.392 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:19.390+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:19.427 INFO:teuthology.orchestra.run.smithi002.stdout:4731 writesame 0x311f5d1 thru 0x312cb89 data_size 0x1(0xd5b9 bytes) 2024-09-22T01:02:19.442 INFO:teuthology.orchestra.run.smithi002.stdout:4733 write 0x5dbb6ec thru 0x5dc63f8 (0xad0d bytes) 2024-09-22T01:02:19.472 INFO:teuthology.orchestra.run.smithi002.stdout:4734 write 0x918351f thru 0x9183c46 (0x728 bytes) 2024-09-22T01:02:19.477 INFO:teuthology.orchestra.run.smithi002.stdout:4735 punch from 0x5f7a80c to 0x5f7f328, (0x4b1c bytes) 2024-09-22T01:02:19.492 INFO:teuthology.orchestra.run.smithi002.stdout:4736 trunc from 0x9183c47 to 0x3084306 2024-09-22T01:02:19.492 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:19.490+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:19.511 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:19.511 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:19.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:4737 writesame 0x4dc854a thru 0x4dc977c data_size 0x1(0x1233 bytes) 2024-09-22T01:02:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:4739 read 0x26a7817 thru 0x26ab4cb (0x3cb5 bytes) 2024-09-22T01:02:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:4741 punch from 0x3b47594 to 0x3b4e2d3, (0x6d3f bytes) 2024-09-22T01:02:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:4743 read 0x1b9ed0c thru 0x1ba63f5 (0x76ea bytes) 2024-09-22T01:02:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:4746 write 0x1fe8460 thru 0x1fec715 (0x42b6 bytes) 2024-09-22T01:02:19.544 INFO:teuthology.orchestra.run.smithi002.stdout:4748 clone 41 order 23 su 32768 sc 6 2024-09-22T01:02:19.593 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:19.590+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:19.655 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523788533000001, 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:02:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:02:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:02:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523732858000002, 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523680513000002, 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.670 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523622602000003, 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:02:19.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:02:19.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523563963000001, 2024-09-22T01:02:19.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:02:19.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:02:19.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:02:19.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523508731, 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:02:19.680 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:02:19.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523455007000003, 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:02:19.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523383271999997, 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:02:19.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:02:19.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523325890999999, 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:19.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:02:19.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523275624999997, 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:02:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:02:19.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:02:19.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523196568000003, 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:02:19.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:02:19.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:02:19.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.523072319000001, 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:19.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:02:19.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.522991736999998, 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.715 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.522926548999997, 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:02:19.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.724 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.522151579999999, 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:19.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.729 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.522101794999998, 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:02:19.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:02:19.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:02:19.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.522020693999998, 2024-09-22T01:02:19.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:02:19.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:02:19.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.521950709000002, 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:02:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:02:19.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:19.744 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.521873345000003, 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:02:19.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:02:19.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:02:19.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.521812824000001, 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:02:19.749 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:19.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:19.754 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:19.755 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:19.756 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:19.754+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:19.856 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:19.854+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:19.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:19.889 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.6 to 1 (10000) 2024-09-22T01:02:19.889 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:02:19.896 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:19.896 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:19.896 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:19.896 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:19.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-22T01:02:19.927 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:19.971 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:19.970+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:20.035 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone40 2024-09-22T01:02:20.045 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:20.046 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:20.058 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:20.058+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:20.077 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:20.077 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:20.109 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:20.109 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:20.126 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:20.126 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:20.134 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:20.135 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:20.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:20.161 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:20.161 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:20.162 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:20.158+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:20.198 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:20.233 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:20.234 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:20.260 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:20.254+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:20.266 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:20.266 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:20.320 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:20.320 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:20.337 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:20.382 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:20.382+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:20.462 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:20.462+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:20.512 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:20.512 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:20.527 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:20.536 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-22T01:02:20.582 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:20.582+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:20.664 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:20.658+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:20.665 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:20.665 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:20.681 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:20.770 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:20.770+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:20.832 INFO:teuthology.orchestra.run.smithi002.stdout:4749 read 0x1ee5bb9 thru 0x1ee7703 (0x1b4b bytes) 2024-09-22T01:02:20.844 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:20.844 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:20.851 INFO:teuthology.orchestra.run.smithi002.stdout:4750 write 0x51434ed thru 0x5143ff8 (0xb0c bytes) 2024-09-22T01:02:20.857 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:20.858 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:20.880 INFO:teuthology.orchestra.run.smithi002.stdout:4753 write 0xe7071d2 thru 0xe712ea8 (0xbcd7 bytes) 2024-09-22T01:02:20.881 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:20.878+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:20.903 INFO:teuthology.orchestra.run.smithi002.stdout:4754 trunc from 0xe712ea9 to 0x4f1c4a3 2024-09-22T01:02:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.7 to 1 (10000) 2024-09-22T01:02:20.926 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T01:02:20.926 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:02:20.947 INFO:teuthology.orchestra.run.smithi002.stdout:4755 read 0x4480135 thru 0x448f366 (0xf232 bytes) 2024-09-22T01:02:20.959 INFO:teuthology.orchestra.run.smithi002.stdout:4756 trunc from 0x4f1c4a3 to 0xe21dc63 2024-09-22T01:02:20.966 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:20.962+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:20.970 INFO:teuthology.orchestra.run.smithi002.stdout:4757 write 0x42ef636 thru 0x42f784c (0x8217 bytes) 2024-09-22T01:02:20.972 INFO:teuthology.orchestra.run.smithi002.stdout:4758 punch from 0x86f7af2 to 0x86fca15, (0x4f23 bytes) 2024-09-22T01:02:20.972 INFO:teuthology.orchestra.run.smithi002.stdout:4760 write 0x496a14b thru 0x4973363 (0x9219 bytes) 2024-09-22T01:02:20.980 INFO:teuthology.orchestra.run.smithi002.stdout:4761 trunc from 0xe21dc63 to 0xaa46f3f 2024-09-22T01:02:21.003 INFO:teuthology.orchestra.run.smithi002.stdout:4764 write 0xc43203c thru 0xc43ece3 (0xcca8 bytes) 2024-09-22T01:02:21.018 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:21.018 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:21.025 INFO:teuthology.orchestra.run.smithi002.stdout:4765 writesame 0xee35c02 thru 0xee3ea12 data_size 0x1b(0x8e11 bytes) 2024-09-22T01:02:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:4769 write 0x5e63862 thru 0x5e66bc4 (0x3363 bytes) 2024-09-22T01:02:21.030 INFO:teuthology.orchestra.run.smithi002.stdout:4770 read 0xdccde34 thru 0xdcd1eab (0x4078 bytes) 2024-09-22T01:02:21.030 INFO:teuthology.orchestra.run.smithi002.stdout:4771 writesame 0x5b4ae13 thru 0x5b5a32c data_size 0x89(0xf51a bytes) 2024-09-22T01:02:21.034 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:21.035 INFO:teuthology.orchestra.run.smithi002.stdout:4772 write 0x7851333 thru 0x7854a9f (0x376d bytes) 2024-09-22T01:02:21.040 INFO:teuthology.orchestra.run.smithi002.stdout:4775 punch from 0x21374eb to 0x213e859, (0x736e bytes) 2024-09-22T01:02:21.040 INFO:teuthology.orchestra.run.smithi002.stdout:4776 write 0xb472aaf thru 0xb481400 (0xe952 bytes) 2024-09-22T01:02:21.049 INFO:teuthology.orchestra.run.smithi002.stdout:4777 write 0x2134a33 thru 0x213e997 (0x9f65 bytes) 2024-09-22T01:02:21.066 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:21.066+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:21.085 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:21.085 INFO:teuthology.orchestra.run.smithi002.stdout:4779 write 0xa29e2cb thru 0xa2a65df (0x8315 bytes) 2024-09-22T01:02:21.091 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:21.091 INFO:teuthology.orchestra.run.smithi002.stdout:4784 punch from 0x794abab to 0x7950e76, (0x62cb bytes) 2024-09-22T01:02:21.091 INFO:teuthology.orchestra.run.smithi002.stdout:4785 read 0x303b710 thru 0x3044104 (0x89f5 bytes) 2024-09-22T01:02:21.092 INFO:teuthology.orchestra.run.smithi002.stdout:4786 write 0x23d4a8e thru 0x23d5bb0 (0x1123 bytes) 2024-09-22T01:02:21.109 INFO:teuthology.orchestra.run.smithi002.stdout:4789 trunc from 0xee3ea13 to 0x3c2ff30 2024-09-22T01:02:21.151 INFO:teuthology.orchestra.run.smithi002.stdout:4790 write 0x8a42e99 thru 0x8a4c2c1 (0x9429 bytes) 2024-09-22T01:02:21.160 INFO:teuthology.orchestra.run.smithi002.stdout:4791 writesame 0x597dedd thru 0x59812a1 data_size 0x1(0x33c5 bytes) 2024-09-22T01:02:21.163 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:21.164 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:21.168 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:21.166+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:21.172 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:21.172 INFO:teuthology.orchestra.run.smithi002.stdout:4795 write 0x4089766 thru 0x4095789 (0xc024 bytes) 2024-09-22T01:02:21.175 INFO:teuthology.orchestra.run.smithi002.stdout:4796 clone 42 order 24 su 2097152 sc 11 2024-09-22T01:02:21.177 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:21.269 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:21.266+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:21.286 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:21.286 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:02:21.304 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-22T01:02:20.906687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195933586,"num_objects":1486,"num_object_clones":418,"num_object_copies":2972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1486,"num_whiteouts":574,"num_read":136499,"num_read_kb":510191,"num_write":16764,"num_write_kb":290095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":60432757,"num_keys_recovered":245,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13413,"ondisk_log_size":13413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":556320,"kb_used_data":314680,"kb_used_omap":13,"kb_used_meta":241458,"kb_avail":660046560,"statfs":{"total":676457349120,"available":675887677440,"internally_reserved":0,"allocated":322232320,"data_stored":394319355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":247253236},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33503221,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":7,"num_read":5255,"num_read_kb":33840,"num_write":785,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027327"},"pg_stats":[{"pgid":"3.1f","version":"97'339","reported_seq":2742,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.625030+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:02:20.625030+0000","last_peered":"2024-09-22T01:02:20.625030+0000","last_clean":"2024-09-22T01:02:20.625030+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:02:20.625030+0000","last_undegraded":"2024-09-22T01:02:20.625030+0000","last_fullsized":"2024-09-22T01:02:20.625030+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:54.837751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":61912890,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":14,"num_read":2454,"num_read_kb":30065,"num_write":390,"num_write_kb":12807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"97'619","reported_seq":2774,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.502809+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:02:20.502809+0000","last_peered":"2024-09-22T01:02:20.502809+0000","last_clean":"2024-09-22T01:02:20.502809+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:02:20.502809+0000","last_undegraded":"2024-09-22T01:02:20.502809+0000","last_fullsized":"2024-09-22T01:02:20.502809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:16:24.292291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":28358594,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":5604,"num_read_kb":11515,"num_write":870,"num_write_kb":6629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"92'346","reported_seq":3656,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879423+0000","last_change":"2024-09-22T01:01:43.138890+0000","last_active":"2024-09-22T01:02:15.879423+0000","last_peered":"2024-09-22T01:02:15.879423+0000","last_clean":"2024-09-22T01:02:15.879423+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:02:15.879423+0000","last_undegraded":"2024-09-22T01:02:15.879423+0000","last_fullsized":"2024-09-22T01:02:15.879423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:38.929152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73059215,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":4032,"num_read_kb":20252,"num_write":375,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"92'856","reported_seq":3812,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879182+0000","last_change":"2024-09-22T01:01:43.141896+0000","last_active":"2024-09-22T01:02:15.879182+0000","last_peered":"2024-09-22T01:02:15.879182+0000","last_clean":"2024-09-22T01:02:15.879182+0000","last_became_active":"2024-09-22T01:01:43.141151+0000","last_became_peered":"2024-09-22T01:01:43.141151+0000","last_unstale":"2024-09-22T01:02:15.879182+0000","last_undegraded":"2024-09-22T01:02:15.879182+0000","last_fullsized":"2024-09-22T01:02:15.879182+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:31:21.780247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61399102,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":19,"num_read":5673,"num_read_kb":34523,"num_write":1288,"num_write_kb":13146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":9399679,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"97'350","reported_seq":1082,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.082758+0000","last_change":"2024-09-22T01:01:43.147132+0000","last_active":"2024-09-22T01:02:20.082758+0000","last_peered":"2024-09-22T01:02:20.082758+0000","last_clean":"2024-09-22T01:02:20.082758+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:02:20.082758+0000","last_undegraded":"2024-09-22T01:02:20.082758+0000","last_fullsized":"2024-09-22T01:02:20.082758+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:45.497251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":26902326,"num_objects":46,"num_object_clones":17,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":1237,"num_read_kb":6633,"num_write":429,"num_write_kb":2121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"96'282","reported_seq":2870,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.514827+0000","last_change":"2024-09-22T01:01:43.143586+0000","last_active":"2024-09-22T01:02:20.514827+0000","last_peered":"2024-09-22T01:02:20.514827+0000","last_clean":"2024-09-22T01:02:20.514827+0000","last_became_active":"2024-09-22T01:01:43.138644+0000","last_became_peered":"2024-09-22T01:01:43.138644+0000","last_unstale":"2024-09-22T01:02:20.514827+0000","last_undegraded":"2024-09-22T01:02:20.514827+0000","last_fullsized":"2024-09-22T01:02:20.514827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:57.294347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":38029861,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":2717,"num_read_kb":6535,"num_write":390,"num_write_kb":7412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":156984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"95'293","reported_seq":4250,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.893840+0000","last_change":"2024-09-22T01:00:49.708786+0000","last_active":"2024-09-22T01:02:19.893840+0000","last_peered":"2024-09-22T01:02:19.893840+0000","last_clean":"2024-09-22T01:02:19.893840+0000","last_became_active":"2024-09-22T01:00:47.723339+0000","last_became_peered":"2024-09-22T01:00:47.723339+0000","last_unstale":"2024-09-22T01:02:19.893840+0000","last_undegraded":"2024-09-22T01:02:19.893840+0000","last_fullsized":"2024-09-22T01:02:19.893840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:11.152631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":29869828,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":11,"num_read":4168,"num_read_kb":19296,"num_write":364,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"97'594","reported_seq":5517,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.469399+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:02:20.469399+0000","last_peered":"2024-09-22T01:02:20.469399+0000","last_clean":"2024-09-22T01:02:20.469399+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:02:20.469399+0000","last_undegraded":"2024-09-22T01:02:20.469399+0000","last_fullsized":"2024-09-22T01:02:20.469399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:40.701309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":76931284,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":14,"num_read":7718,"num_read_kb":32089,"num_write":713,"num_write_kb":9038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"96'410","reported_seq":3390,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.984200+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:02:19.984200+0000","last_peered":"2024-09-22T01:02:19.984200+0000","last_clean":"2024-09-22T01:02:19.984200+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:02:19.984200+0000","last_undegraded":"2024-09-22T01:02:19.984200+0000","last_fullsized":"2024-09-22T01:02:19.984200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:59:25.200613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":20067184,"num_objects":46,"num_object_clones":16,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":4227,"num_read_kb":13106,"num_write":506,"num_write_kb":5547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"93'243","reported_seq":1301,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.996636+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:02:15.996636+0000","last_peered":"2024-09-22T01:02:15.996636+0000","last_clean":"2024-09-22T01:02:15.996636+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:02:15.996636+0000","last_undegraded":"2024-09-22T01:02:15.996636+0000","last_fullsized":"2024-09-22T01:02:15.996636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:00.278335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75466762,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":2153,"num_read_kb":22553,"num_write":286,"num_write_kb":20892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'502","reported_seq":5725,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.516172+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:02:20.516172+0000","last_peered":"2024-09-22T01:02:20.516172+0000","last_clean":"2024-09-22T01:02:20.516172+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:02:20.516172+0000","last_undegraded":"2024-09-22T01:02:20.516172+0000","last_fullsized":"2024-09-22T01:02:20.516172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:28:58.606882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":27796518,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":7890,"num_read_kb":23307,"num_write":564,"num_write_kb":2018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"96'394","reported_seq":8936,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907721+0000","last_change":"2024-09-22T01:01:43.139393+0000","last_active":"2024-09-22T01:02:19.907721+0000","last_peered":"2024-09-22T01:02:19.907721+0000","last_clean":"2024-09-22T01:02:19.907721+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:02:19.907721+0000","last_undegraded":"2024-09-22T01:02:19.907721+0000","last_fullsized":"2024-09-22T01:02:19.907721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:21.878125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":47464584,"num_objects":48,"num_object_clones":16,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":9261,"num_read_kb":24318,"num_write":434,"num_write_kb":10690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.955513+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:02:15.955513+0000","last_peered":"2024-09-22T01:02:15.955513+0000","last_clean":"2024-09-22T01:02:15.955513+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:02:15.955513+0000","last_undegraded":"2024-09-22T01:02:15.955513+0000","last_fullsized":"2024-09-22T01:02:15.955513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"96'388","reported_seq":1847,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.983823+0000","last_change":"2024-09-22T01:00:49.729709+0000","last_active":"2024-09-22T01:02:19.983823+0000","last_peered":"2024-09-22T01:02:19.983823+0000","last_clean":"2024-09-22T01:02:19.983823+0000","last_became_active":"2024-09-22T01:00:47.718964+0000","last_became_peered":"2024-09-22T01:00:47.718964+0000","last_unstale":"2024-09-22T01:02:19.983823+0000","last_undegraded":"2024-09-22T01:02:19.983823+0000","last_fullsized":"2024-09-22T01:02:19.983823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:26:04.423020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":16578833,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":3482,"num_read_kb":4714,"num_write":429,"num_write_kb":1590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.894081+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:02:19.894081+0000","last_peered":"2024-09-22T01:02:19.894081+0000","last_clean":"2024-09-22T01:02:19.894081+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:02:19.894081+0000","last_undegraded":"2024-09-22T01:02:19.894081+0000","last_fullsized":"2024-09-22T01:02:19.894081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"97'433","reported_seq":2423,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.113646+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:02:20.113646+0000","last_peered":"2024-09-22T01:02:20.113646+0000","last_clean":"2024-09-22T01:02:20.113646+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:02:20.113646+0000","last_undegraded":"2024-09-22T01:02:20.113646+0000","last_fullsized":"2024-09-22T01:02:20.113646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:18:22.205254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":34093509,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":2994,"num_read_kb":9089,"num_write":590,"num_write_kb":5671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.908080+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:02:19.908080+0000","last_peered":"2024-09-22T01:02:19.908080+0000","last_clean":"2024-09-22T01:02:19.908080+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:02:19.908080+0000","last_undegraded":"2024-09-22T01:02:19.908080+0000","last_fullsized":"2024-09-22T01:02:19.908080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"97'279","reported_seq":3097,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.495690+0000","last_change":"2024-09-22T01:00:49.709328+0000","last_active":"2024-09-22T01:02:20.495690+0000","last_peered":"2024-09-22T01:02:20.495690+0000","last_clean":"2024-09-22T01:02:20.495690+0000","last_became_active":"2024-09-22T01:00:47.714621+0000","last_became_peered":"2024-09-22T01:00:47.714621+0000","last_unstale":"2024-09-22T01:02:20.495690+0000","last_undegraded":"2024-09-22T01:02:20.495690+0000","last_fullsized":"2024-09-22T01:02:20.495690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:55:06.554976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":15877252,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":3373,"num_read_kb":17927,"num_write":268,"num_write_kb":9231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.893926+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:02:19.893926+0000","last_peered":"2024-09-22T01:02:19.893926+0000","last_clean":"2024-09-22T01:02:19.893926+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:02:19.893926+0000","last_undegraded":"2024-09-22T01:02:19.893926+0000","last_fullsized":"2024-09-22T01:02:19.893926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"93'416","reported_seq":3006,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879426+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:02:15.879426+0000","last_peered":"2024-09-22T01:02:15.879426+0000","last_clean":"2024-09-22T01:02:15.879426+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:02:15.879426+0000","last_undegraded":"2024-09-22T01:02:15.879426+0000","last_fullsized":"2024-09-22T01:02:15.879426+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:07.476956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65651734,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":5007,"num_read_kb":8504,"num_write":389,"num_write_kb":3560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"95'287","reported_seq":1060,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.983424+0000","last_change":"2024-09-22T01:01:43.148315+0000","last_active":"2024-09-22T01:02:19.983424+0000","last_peered":"2024-09-22T01:02:19.983424+0000","last_clean":"2024-09-22T01:02:19.983424+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:02:19.983424+0000","last_undegraded":"2024-09-22T01:02:19.983424+0000","last_fullsized":"2024-09-22T01:02:19.983424+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:30:30.542310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":19383445,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":1182,"num_read_kb":6424,"num_write":319,"num_write_kb":6898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879052+0000","last_change":"2024-09-22T01:01:27.722404+0000","last_active":"2024-09-22T01:02:15.879052+0000","last_peered":"2024-09-22T01:02:15.879052+0000","last_clean":"2024-09-22T01:02:15.879052+0000","last_became_active":"2024-09-22T01:01:27.721384+0000","last_became_peered":"2024-09-22T01:01:27.721384+0000","last_unstale":"2024-09-22T01:02:15.879052+0000","last_undegraded":"2024-09-22T01:02:15.879052+0000","last_fullsized":"2024-09-22T01:02:15.879052+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:10.574832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"97'1484","reported_seq":2781,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.189958+0000","last_change":"2024-09-22T01:00:49.709391+0000","last_active":"2024-09-22T01:02:20.189958+0000","last_peered":"2024-09-22T01:02:20.189958+0000","last_clean":"2024-09-22T01:02:20.189958+0000","last_became_active":"2024-09-22T01:00:47.714544+0000","last_became_peered":"2024-09-22T01:00:47.714544+0000","last_unstale":"2024-09-22T01:02:20.189958+0000","last_undegraded":"2024-09-22T01:02:20.189958+0000","last_fullsized":"2024-09-22T01:02:20.189958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1484,"log_dups_size":0,"ondisk_log_size":1484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:18.478536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":33374480,"num_objects":51,"num_object_clones":17,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":8577,"num_read_kb":18233,"num_write":2565,"num_write_kb":8976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.895967+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:02:19.895967+0000","last_peered":"2024-09-22T01:02:19.895967+0000","last_clean":"2024-09-22T01:02:19.895967+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:02:19.895967+0000","last_undegraded":"2024-09-22T01:02:19.895967+0000","last_fullsized":"2024-09-22T01:02:19.895967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"95'305","reported_seq":2294,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.983297+0000","last_change":"2024-09-22T01:01:43.152419+0000","last_active":"2024-09-22T01:02:19.983297+0000","last_peered":"2024-09-22T01:02:19.983297+0000","last_clean":"2024-09-22T01:02:19.983297+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:02:19.983297+0000","last_undegraded":"2024-09-22T01:02:19.983297+0000","last_fullsized":"2024-09-22T01:02:19.983297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:57:27.874801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":42829827,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":19,"num_read":2051,"num_read_kb":16935,"num_write":341,"num_write_kb":13327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907129+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:02:19.907129+0000","last_peered":"2024-09-22T01:02:19.907129+0000","last_clean":"2024-09-22T01:02:19.907129+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:02:19.907129+0000","last_undegraded":"2024-09-22T01:02:19.907129+0000","last_fullsized":"2024-09-22T01:02:19.907129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"96'277","reported_seq":1664,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.903986+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:02:19.903986+0000","last_peered":"2024-09-22T01:02:19.903986+0000","last_clean":"2024-09-22T01:02:19.903986+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:02:19.903986+0000","last_undegraded":"2024-09-22T01:02:19.903986+0000","last_fullsized":"2024-09-22T01:02:19.903986+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:27:26.568038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":24842243,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":1441,"num_read_kb":14010,"num_write":298,"num_write_kb":16175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":117,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879176+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:02:15.879176+0000","last_peered":"2024-09-22T01:02:15.879176+0000","last_clean":"2024-09-22T01:02:15.879176+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:02:15.879176+0000","last_undegraded":"2024-09-22T01:02:15.879176+0000","last_fullsized":"2024-09-22T01:02:15.879176+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":322,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907243+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:02:19.907243+0000","last_peered":"2024-09-22T01:02:19.907243+0000","last_clean":"2024-09-22T01:02:19.907243+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:02:19.907243+0000","last_undegraded":"2024-09-22T01:02:19.907243+0000","last_fullsized":"2024-09-22T01:02:19.907243+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"97'313","reported_seq":9239,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.087762+0000","last_change":"2024-09-22T01:00:49.735787+0000","last_active":"2024-09-22T01:02:20.087762+0000","last_peered":"2024-09-22T01:02:20.087762+0000","last_clean":"2024-09-22T01:02:20.087762+0000","last_became_active":"2024-09-22T01:00:47.720325+0000","last_became_peered":"2024-09-22T01:00:47.720325+0000","last_unstale":"2024-09-22T01:02:20.087762+0000","last_undegraded":"2024-09-22T01:02:20.087762+0000","last_fullsized":"2024-09-22T01:02:20.087762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:40:59.506494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":47400926,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":10099,"num_read_kb":24462,"num_write":336,"num_write_kb":11597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"97'431","reported_seq":1591,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.071966+0000","last_change":"2024-09-22T01:01:43.142999+0000","last_active":"2024-09-22T01:02:20.071966+0000","last_peered":"2024-09-22T01:02:20.071966+0000","last_clean":"2024-09-22T01:02:20.071966+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:02:20.071966+0000","last_undegraded":"2024-09-22T01:02:20.071966+0000","last_fullsized":"2024-09-22T01:02:20.071966+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:30:03.433498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":29183182,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":2308,"num_read_kb":4780,"num_write":530,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"93'507","reported_seq":10192,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:16.583198+0000","last_change":"2024-09-22T01:01:43.139528+0000","last_active":"2024-09-22T01:02:16.583198+0000","last_peered":"2024-09-22T01:02:16.583198+0000","last_clean":"2024-09-22T01:02:16.583198+0000","last_became_active":"2024-09-22T01:01:43.138786+0000","last_became_peered":"2024-09-22T01:01:43.138786+0000","last_unstale":"2024-09-22T01:02:16.583198+0000","last_undegraded":"2024-09-22T01:02:16.583198+0000","last_fullsized":"2024-09-22T01:02:16.583198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:41:12.006701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":41872958,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":11042,"num_read_kb":17974,"num_write":561,"num_write_kb":9422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":167229,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"95'391","reported_seq":1877,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.904168+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:02:19.904168+0000","last_peered":"2024-09-22T01:02:19.904168+0000","last_clean":"2024-09-22T01:02:19.904168+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:02:19.904168+0000","last_undegraded":"2024-09-22T01:02:19.904168+0000","last_fullsized":"2024-09-22T01:02:19.904168+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:46:20.864982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":19277367,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":3146,"num_read_kb":15695,"num_write":462,"num_write_kb":7452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"90'293","reported_seq":730,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879260+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:02:15.879260+0000","last_peered":"2024-09-22T01:02:15.879260+0000","last_clean":"2024-09-22T01:02:15.879260+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:02:15.879260+0000","last_undegraded":"2024-09-22T01:02:15.879260+0000","last_fullsized":"2024-09-22T01:02:15.879260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:33.395191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27120964,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":2357,"num_read_kb":19447,"num_write":308,"num_write_kb":8441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"97'595","reported_seq":5151,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.194538+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:02:20.194538+0000","last_peered":"2024-09-22T01:02:20.194538+0000","last_clean":"2024-09-22T01:02:20.194538+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:02:20.194538+0000","last_undegraded":"2024-09-22T01:02:20.194538+0000","last_fullsized":"2024-09-22T01:02:20.194538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:00:25.621166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":31600565,"num_objects":57,"num_object_clones":18,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":20,"num_read":6879,"num_read_kb":27537,"num_write":779,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"97'305","reported_seq":3643,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.624382+0000","last_change":"2024-09-22T01:01:43.140327+0000","last_active":"2024-09-22T01:02:20.624382+0000","last_peered":"2024-09-22T01:02:20.624382+0000","last_clean":"2024-09-22T01:02:20.624382+0000","last_became_active":"2024-09-22T01:01:43.139857+0000","last_became_peered":"2024-09-22T01:01:43.139857+0000","last_unstale":"2024-09-22T01:02:20.624382+0000","last_undegraded":"2024-09-22T01:02:20.624382+0000","last_fullsized":"2024-09-22T01:02:20.624382+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:04.014593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":48155678,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":3252,"num_read_kb":23724,"num_write":351,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":2830133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"96'233","reported_seq":3649,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907976+0000","last_change":"2024-09-22T01:01:43.142453+0000","last_active":"2024-09-22T01:02:19.907976+0000","last_peered":"2024-09-22T01:02:19.907976+0000","last_clean":"2024-09-22T01:02:19.907976+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:02:19.907976+0000","last_undegraded":"2024-09-22T01:02:19.907976+0000","last_fullsized":"2024-09-22T01:02:19.907976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:16:36.790607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":23429777,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3929,"num_read_kb":4219,"num_write":220,"num_write_kb":7563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"89'311","reported_seq":2442,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.955352+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:02:15.955352+0000","last_peered":"2024-09-22T01:02:15.955352+0000","last_clean":"2024-09-22T01:02:15.955352+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:02:15.955352+0000","last_undegraded":"2024-09-22T01:02:15.955352+0000","last_fullsized":"2024-09-22T01:02:15.955352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:53:24.731878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":38981771,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":2693,"num_read_kb":11025,"num_write":296,"num_write_kb":9482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"96'349","reported_seq":1294,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907495+0000","last_change":"2024-09-22T01:01:43.139843+0000","last_active":"2024-09-22T01:02:19.907495+0000","last_peered":"2024-09-22T01:02:19.907495+0000","last_clean":"2024-09-22T01:02:19.907495+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:02:19.907495+0000","last_undegraded":"2024-09-22T01:02:19.907495+0000","last_fullsized":"2024-09-22T01:02:19.907495+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:31.014162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":25141013,"num_objects":42,"num_object_clones":16,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":1717,"num_read_kb":5162,"num_write":431,"num_write_kb":2894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"96'260","reported_seq":2400,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.983379+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:02:19.983379+0000","last_peered":"2024-09-22T01:02:19.983379+0000","last_clean":"2024-09-22T01:02:19.983379+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:02:19.983379+0000","last_undegraded":"2024-09-22T01:02:19.983379+0000","last_fullsized":"2024-09-22T01:02:19.983379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:03:37.843363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":24020824,"num_objects":43,"num_object_clones":15,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":2833,"num_read_kb":5920,"num_write":242,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"97'208","reported_seq":1375,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.083994+0000","last_change":"2024-09-22T01:02:06.801024+0000","last_active":"2024-09-22T01:02:20.083994+0000","last_peered":"2024-09-22T01:02:20.083994+0000","last_clean":"2024-09-22T01:02:20.083994+0000","last_became_active":"2024-09-22T01:02:06.799165+0000","last_became_peered":"2024-09-22T01:02:06.799165+0000","last_unstale":"2024-09-22T01:02:20.083994+0000","last_undegraded":"2024-09-22T01:02:20.083994+0000","last_fullsized":"2024-09-22T01:02:20.083994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:31:04.205320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":19268703,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":887,"num_read_kb":10118,"num_write":279,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":11755308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1195343199,"num_objects":1482,"num_object_clones":418,"num_object_copies":2964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1482,"num_whiteouts":574,"num_read":136383,"num_read_kb":510091,"num_write":16603,"num_write_kb":287677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":60432757,"num_keys_recovered":245,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":341516288,"data_stored":423551911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":13293,"ondisk_log_size":13293,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70844,"kb_used_data":35492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94300996,"statfs":{"total":96636764160,"available":96564219904,"internally_reserved":0,"allocated":36343808,"data_stored":43937341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299403,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100632,"kb_used_data":64200,"kb_used_omap":4,"kb_used_meta":36411,"kb_avail":94271208,"statfs":{"total":96636764160,"available":96533716992,"internally_reserved":0,"allocated":65740800,"data_stored":86074594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":37285454},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67972,"kb_used_data":40948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94303868,"statfs":{"total":96636764160,"available":96567160832,"internally_reserved":0,"allocated":41930752,"data_stored":50643775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542170,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96444,"kb_used_data":61088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94275396,"statfs":{"total":96636764160,"available":96538005504,"internally_reserved":0,"allocated":62554112,"data_stored":75007234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48680,"kb_used_data":21656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323160,"statfs":{"total":96636764160,"available":96586915840,"internally_reserved":0,"allocated":22175744,"data_stored":27774835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91356,"kb_used_data":42124,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94280484,"statfs":{"total":96636764160,"available":96543215616,"internally_reserved":0,"allocated":43134976,"data_stored":53538059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542170,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80392,"kb_used_data":49172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94291448,"statfs":{"total":96636764160,"available":96554442752,"internally_reserved":0,"allocated":50352128,"data_stored":57343517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":48615424,"data_stored":56222268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":53007159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21032960,"data_stored":27243954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":62984192,"data_stored":77622081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":64790528,"data_stored":85692163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":35119104,"data_stored":43305763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40194048,"data_stored":49522526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:02:21.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:02:21.337 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:21.337 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:21.353 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:21.398 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:21.397+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:21.470 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:21.466+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:21.526 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:21.526 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:21.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:21.571 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:21.570+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:21.639 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:02:21.655 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:21.655 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:21.656 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-22T01:02:20.906687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195933586,"num_objects":1486,"num_object_clones":418,"num_object_copies":2972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1486,"num_whiteouts":574,"num_read":136499,"num_read_kb":510191,"num_write":16764,"num_write_kb":290095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":60432757,"num_keys_recovered":245,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13413,"ondisk_log_size":13413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":556320,"kb_used_data":314680,"kb_used_omap":13,"kb_used_meta":241458,"kb_avail":660046560,"statfs":{"total":676457349120,"available":675887677440,"internally_reserved":0,"allocated":322232320,"data_stored":394319355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":247253236},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33503221,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":7,"num_read":5255,"num_read_kb":33840,"num_write":785,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027327"},"pg_stats":[{"pgid":"3.1f","version":"97'339","reported_seq":2742,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.625030+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:02:20.625030+0000","last_peered":"2024-09-22T01:02:20.625030+0000","last_clean":"2024-09-22T01:02:20.625030+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:02:20.625030+0000","last_undegraded":"2024-09-22T01:02:20.625030+0000","last_fullsized":"2024-09-22T01:02:20.625030+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:54.837751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":61912890,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":14,"num_read":2454,"num_read_kb":30065,"num_write":390,"num_write_kb":12807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"97'619","reported_seq":2774,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.502809+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:02:20.502809+0000","last_peered":"2024-09-22T01:02:20.502809+0000","last_clean":"2024-09-22T01:02:20.502809+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:02:20.502809+0000","last_undegraded":"2024-09-22T01:02:20.502809+0000","last_fullsized":"2024-09-22T01:02:20.502809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:16:24.292291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":28358594,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":5604,"num_read_kb":11515,"num_write":870,"num_write_kb":6629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"92'346","reported_seq":3656,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879423+0000","last_change":"2024-09-22T01:01:43.138890+0000","last_active":"2024-09-22T01:02:15.879423+0000","last_peered":"2024-09-22T01:02:15.879423+0000","last_clean":"2024-09-22T01:02:15.879423+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:02:15.879423+0000","last_undegraded":"2024-09-22T01:02:15.879423+0000","last_fullsized":"2024-09-22T01:02:15.879423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:38.929152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73059215,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":4032,"num_read_kb":20252,"num_write":375,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"92'856","reported_seq":3812,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879182+0000","last_change":"2024-09-22T01:01:43.141896+0000","last_active":"2024-09-22T01:02:15.879182+0000","last_peered":"2024-09-22T01:02:15.879182+0000","last_clean":"2024-09-22T01:02:15.879182+0000","last_became_active":"2024-09-22T01:01:43.141151+0000","last_became_peered":"2024-09-22T01:01:43.141151+0000","last_unstale":"2024-09-22T01:02:15.879182+0000","last_undegraded":"2024-09-22T01:02:15.879182+0000","last_fullsized":"2024-09-22T01:02:15.879182+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:31:21.780247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61399102,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":19,"num_read":5673,"num_read_kb":34523,"num_write":1288,"num_write_kb":13146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":9399679,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"97'350","reported_seq":1082,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.082758+0000","last_change":"2024-09-22T01:01:43.147132+0000","last_active":"2024-09-22T01:02:20.082758+0000","last_peered":"2024-09-22T01:02:20.082758+0000","last_clean":"2024-09-22T01:02:20.082758+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:02:20.082758+0000","last_undegraded":"2024-09-22T01:02:20.082758+0000","last_fullsized":"2024-09-22T01:02:20.082758+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:45.497251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":26902326,"num_objects":46,"num_object_clones":17,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":1237,"num_read_kb":6633,"num_write":429,"num_write_kb":2121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"96'282","reported_seq":2870,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.514827+0000","last_change":"2024-09-22T01:01:43.143586+0000","last_active":"2024-09-22T01:02:20.514827+0000","last_peered":"2024-09-22T01:02:20.514827+0000","last_clean":"2024-09-22T01:02:20.514827+0000","last_became_active":"2024-09-22T01:01:43.138644+0000","last_became_peered":"2024-09-22T01:01:43.138644+0000","last_unstale":"2024-09-22T01:02:20.514827+0000","last_undegraded":"2024-09-22T01:02:20.514827+0000","last_fullsized":"2024-09-22T01:02:20.514827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:57.294347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":38029861,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":2717,"num_read_kb":6535,"num_write":390,"num_write_kb":7412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":156984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"95'293","reported_seq":4250,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.893840+0000","last_change":"2024-09-22T01:00:49.708786+0000","last_active":"2024-09-22T01:02:19.893840+0000","last_peered":"2024-09-22T01:02:19.893840+0000","last_clean":"2024-09-22T01:02:19.893840+0000","last_became_active":"2024-09-22T01:00:47.723339+0000","last_became_peered":"2024-09-22T01:00:47.723339+0000","last_unstale":"2024-09-22T01:02:19.893840+0000","last_undegraded":"2024-09-22T01:02:19.893840+0000","last_fullsized":"2024-09-22T01:02:19.893840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:11.152631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":29869828,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":11,"num_read":4168,"num_read_kb":19296,"num_write":364,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"97'594","reported_seq":5517,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.469399+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:02:20.469399+0000","last_peered":"2024-09-22T01:02:20.469399+0000","last_clean":"2024-09-22T01:02:20.469399+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:02:20.469399+0000","last_undegraded":"2024-09-22T01:02:20.469399+0000","last_fullsized":"2024-09-22T01:02:20.469399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:40.701309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":76931284,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":14,"num_read":7718,"num_read_kb":32089,"num_write":713,"num_write_kb":9038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"96'410","reported_seq":3390,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.984200+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:02:19.984200+0000","last_peered":"2024-09-22T01:02:19.984200+0000","last_clean":"2024-09-22T01:02:19.984200+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:02:19.984200+0000","last_undegraded":"2024-09-22T01:02:19.984200+0000","last_fullsized":"2024-09-22T01:02:19.984200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:59:25.200613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":20067184,"num_objects":46,"num_object_clones":16,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":4227,"num_read_kb":13106,"num_write":506,"num_write_kb":5547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"93'243","reported_seq":1301,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.996636+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:02:15.996636+0000","last_peered":"2024-09-22T01:02:15.996636+0000","last_clean":"2024-09-22T01:02:15.996636+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:02:15.996636+0000","last_undegraded":"2024-09-22T01:02:15.996636+0000","last_fullsized":"2024-09-22T01:02:15.996636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:00.278335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75466762,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":2153,"num_read_kb":22553,"num_write":286,"num_write_kb":20892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'502","reported_seq":5725,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.516172+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:02:20.516172+0000","last_peered":"2024-09-22T01:02:20.516172+0000","last_clean":"2024-09-22T01:02:20.516172+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:02:20.516172+0000","last_undegraded":"2024-09-22T01:02:20.516172+0000","last_fullsized":"2024-09-22T01:02:20.516172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:28:58.606882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":27796518,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":7890,"num_read_kb":23307,"num_write":564,"num_write_kb":2018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"96'394","reported_seq":8936,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907721+0000","last_change":"2024-09-22T01:01:43.139393+0000","last_active":"2024-09-22T01:02:19.907721+0000","last_peered":"2024-09-22T01:02:19.907721+0000","last_clean":"2024-09-22T01:02:19.907721+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:02:19.907721+0000","last_undegraded":"2024-09-22T01:02:19.907721+0000","last_fullsized":"2024-09-22T01:02:19.907721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:21.878125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":47464584,"num_objects":48,"num_object_clones":16,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":9261,"num_read_kb":24318,"num_write":434,"num_write_kb":10690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.955513+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:02:15.955513+0000","last_peered":"2024-09-22T01:02:15.955513+0000","last_clean":"2024-09-22T01:02:15.955513+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:02:15.955513+0000","last_undegraded":"2024-09-22T01:02:15.955513+0000","last_fullsized":"2024-09-22T01:02:15.955513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"96'388","reported_seq":1847,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.983823+0000","last_change":"2024-09-22T01:00:49.729709+0000","last_active":"2024-09-22T01:02:19.983823+0000","last_peered":"2024-09-22T01:02:19.983823+0000","last_clean":"2024-09-22T01:02:19.983823+0000","last_became_active":"2024-09-22T01:00:47.718964+0000","last_became_peered":"2024-09-22T01:00:47.718964+0000","last_unstale":"2024-09-22T01:02:19.983823+0000","last_undegraded":"2024-09-22T01:02:19.983823+0000","last_fullsized":"2024-09-22T01:02:19.983823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:26:04.423020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":16578833,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":3482,"num_read_kb":4714,"num_write":429,"num_write_kb":1590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.894081+0000","last_change":"2024-09-22T01:00:39.946521+0000","last_active":"2024-09-22T01:02:19.894081+0000","last_peered":"2024-09-22T01:02:19.894081+0000","last_clean":"2024-09-22T01:02:19.894081+0000","last_became_active":"2024-09-22T01:00:37.926988+0000","last_became_peered":"2024-09-22T01:00:37.926988+0000","last_unstale":"2024-09-22T01:02:19.894081+0000","last_undegraded":"2024-09-22T01:02:19.894081+0000","last_fullsized":"2024-09-22T01:02:19.894081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:06:23.799063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217667,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"97'433","reported_seq":2423,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.113646+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:02:20.113646+0000","last_peered":"2024-09-22T01:02:20.113646+0000","last_clean":"2024-09-22T01:02:20.113646+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:02:20.113646+0000","last_undegraded":"2024-09-22T01:02:20.113646+0000","last_fullsized":"2024-09-22T01:02:20.113646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:18:22.205254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":34093509,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":2994,"num_read_kb":9089,"num_write":590,"num_write_kb":5671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.908080+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:02:19.908080+0000","last_peered":"2024-09-22T01:02:19.908080+0000","last_clean":"2024-09-22T01:02:19.908080+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:02:19.908080+0000","last_undegraded":"2024-09-22T01:02:19.908080+0000","last_fullsized":"2024-09-22T01:02:19.908080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"97'279","reported_seq":3097,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.495690+0000","last_change":"2024-09-22T01:00:49.709328+0000","last_active":"2024-09-22T01:02:20.495690+0000","last_peered":"2024-09-22T01:02:20.495690+0000","last_clean":"2024-09-22T01:02:20.495690+0000","last_became_active":"2024-09-22T01:00:47.714621+0000","last_became_peered":"2024-09-22T01:00:47.714621+0000","last_unstale":"2024-09-22T01:02:20.495690+0000","last_undegraded":"2024-09-22T01:02:20.495690+0000","last_fullsized":"2024-09-22T01:02:20.495690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:55:06.554976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":15877252,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":3373,"num_read_kb":17927,"num_write":268,"num_write_kb":9231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.893926+0000","last_change":"2024-09-22T01:00:39.946358+0000","last_active":"2024-09-22T01:02:19.893926+0000","last_peered":"2024-09-22T01:02:19.893926+0000","last_clean":"2024-09-22T01:02:19.893926+0000","last_became_active":"2024-09-22T01:00:37.922978+0000","last_became_peered":"2024-09-22T01:00:37.922978+0000","last_unstale":"2024-09-22T01:02:19.893926+0000","last_undegraded":"2024-09-22T01:02:19.893926+0000","last_fullsized":"2024-09-22T01:02:19.893926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:50.710980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011208360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"93'416","reported_seq":3006,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879426+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:02:15.879426+0000","last_peered":"2024-09-22T01:02:15.879426+0000","last_clean":"2024-09-22T01:02:15.879426+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:02:15.879426+0000","last_undegraded":"2024-09-22T01:02:15.879426+0000","last_fullsized":"2024-09-22T01:02:15.879426+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:07.476956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65651734,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":5007,"num_read_kb":8504,"num_write":389,"num_write_kb":3560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"95'287","reported_seq":1060,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.983424+0000","last_change":"2024-09-22T01:01:43.148315+0000","last_active":"2024-09-22T01:02:19.983424+0000","last_peered":"2024-09-22T01:02:19.983424+0000","last_clean":"2024-09-22T01:02:19.983424+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:02:19.983424+0000","last_undegraded":"2024-09-22T01:02:19.983424+0000","last_fullsized":"2024-09-22T01:02:19.983424+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:30:30.542310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":19383445,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":1182,"num_read_kb":6424,"num_write":319,"num_write_kb":6898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879052+0000","last_change":"2024-09-22T01:01:27.722404+0000","last_active":"2024-09-22T01:02:15.879052+0000","last_peered":"2024-09-22T01:02:15.879052+0000","last_clean":"2024-09-22T01:02:15.879052+0000","last_became_active":"2024-09-22T01:01:27.721384+0000","last_became_peered":"2024-09-22T01:01:27.721384+0000","last_unstale":"2024-09-22T01:02:15.879052+0000","last_undegraded":"2024-09-22T01:02:15.879052+0000","last_fullsized":"2024-09-22T01:02:15.879052+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:10.574832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"97'1484","reported_seq":2781,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.189958+0000","last_change":"2024-09-22T01:00:49.709391+0000","last_active":"2024-09-22T01:02:20.189958+0000","last_peered":"2024-09-22T01:02:20.189958+0000","last_clean":"2024-09-22T01:02:20.189958+0000","last_became_active":"2024-09-22T01:00:47.714544+0000","last_became_peered":"2024-09-22T01:00:47.714544+0000","last_unstale":"2024-09-22T01:02:20.189958+0000","last_undegraded":"2024-09-22T01:02:20.189958+0000","last_fullsized":"2024-09-22T01:02:20.189958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1484,"log_dups_size":0,"ondisk_log_size":1484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:18.478536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":33374480,"num_objects":51,"num_object_clones":17,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":8577,"num_read_kb":18233,"num_write":2565,"num_write_kb":8976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.895967+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:02:19.895967+0000","last_peered":"2024-09-22T01:02:19.895967+0000","last_clean":"2024-09-22T01:02:19.895967+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:02:19.895967+0000","last_undegraded":"2024-09-22T01:02:19.895967+0000","last_fullsized":"2024-09-22T01:02:19.895967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"95'305","reported_seq":2294,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.983297+0000","last_change":"2024-09-22T01:01:43.152419+0000","last_active":"2024-09-22T01:02:19.983297+0000","last_peered":"2024-09-22T01:02:19.983297+0000","last_clean":"2024-09-22T01:02:19.983297+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:02:19.983297+0000","last_undegraded":"2024-09-22T01:02:19.983297+0000","last_fullsized":"2024-09-22T01:02:19.983297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:57:27.874801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":42829827,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":19,"num_read":2051,"num_read_kb":16935,"num_write":341,"num_write_kb":13327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907129+0000","last_change":"2024-09-22T01:00:40.354460+0000","last_active":"2024-09-22T01:02:19.907129+0000","last_peered":"2024-09-22T01:02:19.907129+0000","last_clean":"2024-09-22T01:02:19.907129+0000","last_became_active":"2024-09-22T01:00:37.930909+0000","last_became_peered":"2024-09-22T01:00:37.930909+0000","last_unstale":"2024-09-22T01:02:19.907129+0000","last_undegraded":"2024-09-22T01:02:19.907129+0000","last_fullsized":"2024-09-22T01:02:19.907129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:17:37.440773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"96'277","reported_seq":1664,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.903986+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:02:19.903986+0000","last_peered":"2024-09-22T01:02:19.903986+0000","last_clean":"2024-09-22T01:02:19.903986+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:02:19.903986+0000","last_undegraded":"2024-09-22T01:02:19.903986+0000","last_fullsized":"2024-09-22T01:02:19.903986+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:27:26.568038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":24842243,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":1441,"num_read_kb":14010,"num_write":298,"num_write_kb":16175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":117,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879176+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:02:15.879176+0000","last_peered":"2024-09-22T01:02:15.879176+0000","last_clean":"2024-09-22T01:02:15.879176+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:02:15.879176+0000","last_undegraded":"2024-09-22T01:02:15.879176+0000","last_fullsized":"2024-09-22T01:02:15.879176+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":322,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907243+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:02:19.907243+0000","last_peered":"2024-09-22T01:02:19.907243+0000","last_clean":"2024-09-22T01:02:19.907243+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:02:19.907243+0000","last_undegraded":"2024-09-22T01:02:19.907243+0000","last_fullsized":"2024-09-22T01:02:19.907243+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"97'313","reported_seq":9239,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.087762+0000","last_change":"2024-09-22T01:00:49.735787+0000","last_active":"2024-09-22T01:02:20.087762+0000","last_peered":"2024-09-22T01:02:20.087762+0000","last_clean":"2024-09-22T01:02:20.087762+0000","last_became_active":"2024-09-22T01:00:47.720325+0000","last_became_peered":"2024-09-22T01:00:47.720325+0000","last_unstale":"2024-09-22T01:02:20.087762+0000","last_undegraded":"2024-09-22T01:02:20.087762+0000","last_fullsized":"2024-09-22T01:02:20.087762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:40:59.506494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":47400926,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":10099,"num_read_kb":24462,"num_write":336,"num_write_kb":11597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"97'431","reported_seq":1591,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.071966+0000","last_change":"2024-09-22T01:01:43.142999+0000","last_active":"2024-09-22T01:02:20.071966+0000","last_peered":"2024-09-22T01:02:20.071966+0000","last_clean":"2024-09-22T01:02:20.071966+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:02:20.071966+0000","last_undegraded":"2024-09-22T01:02:20.071966+0000","last_fullsized":"2024-09-22T01:02:20.071966+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:30:03.433498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":29183182,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":2308,"num_read_kb":4780,"num_write":530,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"93'507","reported_seq":10192,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:16.583198+0000","last_change":"2024-09-22T01:01:43.139528+0000","last_active":"2024-09-22T01:02:16.583198+0000","last_peered":"2024-09-22T01:02:16.583198+0000","last_clean":"2024-09-22T01:02:16.583198+0000","last_became_active":"2024-09-22T01:01:43.138786+0000","last_became_peered":"2024-09-22T01:01:43.138786+0000","last_unstale":"2024-09-22T01:02:16.583198+0000","last_undegraded":"2024-09-22T01:02:16.583198+0000","last_fullsized":"2024-09-22T01:02:16.583198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:41:12.006701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":41872958,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":11042,"num_read_kb":17974,"num_write":561,"num_write_kb":9422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":167229,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"95'391","reported_seq":1877,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.904168+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:02:19.904168+0000","last_peered":"2024-09-22T01:02:19.904168+0000","last_clean":"2024-09-22T01:02:19.904168+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:02:19.904168+0000","last_undegraded":"2024-09-22T01:02:19.904168+0000","last_fullsized":"2024-09-22T01:02:19.904168+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:46:20.864982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":19277367,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":3146,"num_read_kb":15695,"num_write":462,"num_write_kb":7452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"90'293","reported_seq":730,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.879260+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:02:15.879260+0000","last_peered":"2024-09-22T01:02:15.879260+0000","last_clean":"2024-09-22T01:02:15.879260+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:02:15.879260+0000","last_undegraded":"2024-09-22T01:02:15.879260+0000","last_fullsized":"2024-09-22T01:02:15.879260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:33.395191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27120964,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":2357,"num_read_kb":19447,"num_write":308,"num_write_kb":8441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"97'595","reported_seq":5151,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.194538+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:02:20.194538+0000","last_peered":"2024-09-22T01:02:20.194538+0000","last_clean":"2024-09-22T01:02:20.194538+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:02:20.194538+0000","last_undegraded":"2024-09-22T01:02:20.194538+0000","last_fullsized":"2024-09-22T01:02:20.194538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:00:25.621166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":31600565,"num_objects":57,"num_object_clones":18,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":20,"num_read":6879,"num_read_kb":27537,"num_write":779,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"97'305","reported_seq":3643,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.624382+0000","last_change":"2024-09-22T01:01:43.140327+0000","last_active":"2024-09-22T01:02:20.624382+0000","last_peered":"2024-09-22T01:02:20.624382+0000","last_clean":"2024-09-22T01:02:20.624382+0000","last_became_active":"2024-09-22T01:01:43.139857+0000","last_became_peered":"2024-09-22T01:01:43.139857+0000","last_unstale":"2024-09-22T01:02:20.624382+0000","last_undegraded":"2024-09-22T01:02:20.624382+0000","last_fullsized":"2024-09-22T01:02:20.624382+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:04.014593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":48155678,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":3252,"num_read_kb":23724,"num_write":351,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":2830133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"96'233","reported_seq":3649,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907976+0000","last_change":"2024-09-22T01:01:43.142453+0000","last_active":"2024-09-22T01:02:19.907976+0000","last_peered":"2024-09-22T01:02:19.907976+0000","last_clean":"2024-09-22T01:02:19.907976+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:02:19.907976+0000","last_undegraded":"2024-09-22T01:02:19.907976+0000","last_fullsized":"2024-09-22T01:02:19.907976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:16:36.790607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":23429777,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3929,"num_read_kb":4219,"num_write":220,"num_write_kb":7563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"89'311","reported_seq":2442,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T01:02:15.955352+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:02:15.955352+0000","last_peered":"2024-09-22T01:02:15.955352+0000","last_clean":"2024-09-22T01:02:15.955352+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:02:15.955352+0000","last_undegraded":"2024-09-22T01:02:15.955352+0000","last_fullsized":"2024-09-22T01:02:15.955352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:53:24.731878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":38981771,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":2693,"num_read_kb":11025,"num_write":296,"num_write_kb":9482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"96'349","reported_seq":1294,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.907495+0000","last_change":"2024-09-22T01:01:43.139843+0000","last_active":"2024-09-22T01:02:19.907495+0000","last_peered":"2024-09-22T01:02:19.907495+0000","last_clean":"2024-09-22T01:02:19.907495+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:02:19.907495+0000","last_undegraded":"2024-09-22T01:02:19.907495+0000","last_fullsized":"2024-09-22T01:02:19.907495+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:31.014162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":25141013,"num_objects":42,"num_object_clones":16,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":1717,"num_read_kb":5162,"num_write":431,"num_write_kb":2894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"96'260","reported_seq":2400,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:19.983379+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:02:19.983379+0000","last_peered":"2024-09-22T01:02:19.983379+0000","last_clean":"2024-09-22T01:02:19.983379+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:02:19.983379+0000","last_undegraded":"2024-09-22T01:02:19.983379+0000","last_fullsized":"2024-09-22T01:02:19.983379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:03:37.843363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":24020824,"num_objects":43,"num_object_clones":15,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":2833,"num_read_kb":5920,"num_write":242,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"97'208","reported_seq":1375,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-22T01:02:20.083994+0000","last_change":"2024-09-22T01:02:06.801024+0000","last_active":"2024-09-22T01:02:20.083994+0000","last_peered":"2024-09-22T01:02:20.083994+0000","last_clean":"2024-09-22T01:02:20.083994+0000","last_became_active":"2024-09-22T01:02:06.799165+0000","last_became_peered":"2024-09-22T01:02:06.799165+0000","last_unstale":"2024-09-22T01:02:20.083994+0000","last_undegraded":"2024-09-22T01:02:20.083994+0000","last_fullsized":"2024-09-22T01:02:20.083994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:31:04.205320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":19268703,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":887,"num_read_kb":10118,"num_write":279,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":11755308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1195343199,"num_objects":1482,"num_object_clones":418,"num_object_copies":2964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1482,"num_whiteouts":574,"num_read":136383,"num_read_kb":510091,"num_write":16603,"num_write_kb":287677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":60432757,"num_keys_recovered":245,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":341516288,"data_stored":423551911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":13293,"ondisk_log_size":13293,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70844,"kb_used_data":35492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94300996,"statfs":{"total":96636764160,"available":96564219904,"internally_reserved":0,"allocated":36343808,"data_stored":43937341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299403,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100632,"kb_used_data":64200,"kb_used_omap":4,"kb_used_meta":36411,"kb_avail":94271208,"statfs":{"total":96636764160,"available":96533716992,"internally_reserved":0,"allocated":65740800,"data_stored":86074594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":37285454},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67972,"kb_used_data":40948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94303868,"statfs":{"total":96636764160,"available":96567160832,"internally_reserved":0,"allocated":41930752,"data_stored":50643775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542170,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96444,"kb_used_data":61088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94275396,"statfs":{"total":96636764160,"available":96538005504,"internally_reserved":0,"allocated":62554112,"data_stored":75007234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48680,"kb_used_data":21656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323160,"statfs":{"total":96636764160,"available":96586915840,"internally_reserved":0,"allocated":22175744,"data_stored":27774835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91356,"kb_used_data":42124,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94280484,"statfs":{"total":96636764160,"available":96543215616,"internally_reserved":0,"allocated":43134976,"data_stored":53538059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542170,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80392,"kb_used_data":49172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94291448,"statfs":{"total":96636764160,"available":96554442752,"internally_reserved":0,"allocated":50352128,"data_stored":57343517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":48615424,"data_stored":56222268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":53007159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21032960,"data_stored":27243954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":62984192,"data_stored":77622081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":64790528,"data_stored":85692163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":35119104,"data_stored":43305763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40194048,"data_stored":49522526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:02:21.658 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T01:02:21.658 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:02:21.658 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 5, 6, 7] 2024-09-22T01:02:21.658 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-22T01:02:21.669 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:21.672 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:21.666+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:21.773 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:21.770+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:21.832 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:21.832 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:21.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:21.873 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:21.869+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:21.929 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:02:21.959 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-22T01:02:21.957+0000 7fa2a81e5640 -1 mon.a@0(leader).osd e99 definitely_dead 0 2024-09-22T01:02:21.974 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:21.973+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:21.983 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:21.983 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:21.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone41 from 0x8a4c2c2 (overlap 0x3c2ff30) to 0x2a21908 2024-09-22T01:02:22.074 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:22.070+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:22.135 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:22.136 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:22.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:22.158 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-22T01:02:22.193 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:22.190+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:22.276 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:22.273+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:22.289 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:22.316 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:22.316 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:22.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:22.349 INFO:teuthology.orchestra.run.smithi002.stdout:4797 write 0x9bb35cb thru 0x9bc019d (0xcbd3 bytes) 2024-09-22T01:02:22.367 INFO:teuthology.orchestra.run.smithi002.stdout:4798 read 0x72f6a7e thru 0x72ff136 (0x86b9 bytes) 2024-09-22T01:02:22.367 INFO:teuthology.orchestra.run.smithi002.stdout:4799 write 0x3885ce9 thru 0x3894395 (0xe6ad bytes) 2024-09-22T01:02:22.376 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:22.374+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:22.380 INFO:teuthology.orchestra.run.smithi002.stdout:4801 punch from 0x5f1fe09 to 0x5f26d8d, (0x6f84 bytes) 2024-09-22T01:02:22.380 INFO:teuthology.orchestra.run.smithi002.stdout:4802 write 0x6f901 thru 0x783ce (0x8ace bytes) 2024-09-22T01:02:22.384 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.385 INFO:teuthology.orchestra.run.smithi002.stdout:4806 read 0x6c310d6 thru 0x6c3fba6 (0xead1 bytes) 2024-09-22T01:02:22.385 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.385 INFO:teuthology.orchestra.run.smithi002.stdout:4809 read 0x3f36ab1 thru 0x3f46434 (0xf984 bytes) 2024-09-22T01:02:22.391 INFO:teuthology.orchestra.run.smithi002.stdout:4811 trunc from 0x9bc019e to 0xdf3c85b 2024-09-22T01:02:22.398 INFO:teuthology.orchestra.run.smithi002.stdout:4814 punch from 0x8e5454e to 0x8e5e67d, (0xa12f bytes) 2024-09-22T01:02:22.398 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.407 INFO:teuthology.orchestra.run.smithi002.stdout:4816 trunc from 0xdf3c85b to 0x6cc6713 2024-09-22T01:02:22.448 INFO:teuthology.orchestra.run.smithi002.stdout:4817 read 0x1329c8c thru 0x132db41 (0x3eb6 bytes) 2024-09-22T01:02:22.448 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:22.449 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:22.463 INFO:teuthology.orchestra.run.smithi002.stdout:4819 write 0xcbb2511 thru 0xcbc1dfb (0xf8eb bytes) 2024-09-22T01:02:22.466 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:22.467 INFO:teuthology.orchestra.run.smithi002.stdout:4821 writesame 0x30c464 thru 0x30d4bf data_size 0x6(0x105c bytes) 2024-09-22T01:02:22.467 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:22.467 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:22.469 INFO:teuthology.orchestra.run.smithi002.stdout:4822 punch from 0x8daf9b4 to 0x8db9883, (0x9ecf bytes) 2024-09-22T01:02:22.469 INFO:teuthology.orchestra.run.smithi002.stdout:4823 write 0x5ae58ae thru 0x5af4182 (0xe8d5 bytes) 2024-09-22T01:02:22.470 INFO:teuthology.orchestra.run.smithi002.stdout:4824 punch from 0x2278d85 to 0x227e6e1, (0x595c bytes) 2024-09-22T01:02:22.471 INFO:teuthology.orchestra.run.smithi002.stdout:4825 read 0xc7be820 thru 0xc7cc94c (0xe12d bytes) 2024-09-22T01:02:22.471 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.471 INFO:teuthology.orchestra.run.smithi002.stdout:4827 read 0xaaf3546 thru 0xaaf65be (0x3079 bytes) 2024-09-22T01:02:22.471 INFO:teuthology.orchestra.run.smithi002.stdout:4828 writesame 0xba6c200 thru 0xba6dbcf data_size 0x38(0x19d0 bytes) 2024-09-22T01:02:22.474 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.474 INFO:teuthology.orchestra.run.smithi002.stdout:4832 writesame 0x59769bb thru 0x5982a1b data_size 0x1(0xc061 bytes) 2024-09-22T01:02:22.477 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:22.474+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:22.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:22.484 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:22.500 INFO:teuthology.orchestra.run.smithi002.stdout:4833 write 0x62ad872 thru 0x62b8e37 (0xb5c6 bytes) 2024-09-22T01:02:22.501 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:22.501 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:22.512 INFO:teuthology.orchestra.run.smithi002.stdout:4834 trunc from 0xcbc1dfc to 0x4a98e31 2024-09-22T01:02:22.522 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:22.526 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.526 INFO:teuthology.orchestra.run.smithi002.stdout:4836 read 0x848f0a thru 0x85508f (0xc186 bytes) 2024-09-22T01:02:22.526 INFO:teuthology.orchestra.run.smithi002.stdout:4837 write 0x110e00a thru 0x111b4a7 (0xd49e bytes) 2024-09-22T01:02:22.540 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:22.542 INFO:teuthology.orchestra.run.smithi002.stdout:4839 write 0xb46c757 thru 0xb46d2f8 (0xba2 bytes) 2024-09-22T01:02:22.546 INFO:teuthology.orchestra.run.smithi002.stdout:4840 read 0x3fe3fd3 thru 0x3ff2b34 (0xeb62 bytes) 2024-09-22T01:02:22.546 INFO:teuthology.orchestra.run.smithi002.stdout:4841 punch from 0x234bb90 to 0x234c5c3, (0xa33 bytes) 2024-09-22T01:02:22.547 INFO:teuthology.orchestra.run.smithi002.stdout:4842 write 0xa5a8354 thru 0xa5af68a (0x7337 bytes) 2024-09-22T01:02:22.549 INFO:teuthology.orchestra.run.smithi002.stdout:4843 read 0x3efd36a thru 0x3efdd6d (0xa04 bytes) 2024-09-22T01:02:22.549 INFO:teuthology.orchestra.run.smithi002.stdout:4844 punch from 0x7a6b7d2 to 0x7a71d58, (0x6586 bytes) 2024-09-22T01:02:22.558 INFO:teuthology.orchestra.run.smithi002.stdout:4845 trunc from 0xb46d2f9 to 0x3627102 2024-09-22T01:02:22.558 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:22.559 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:22.578 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:22.574+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:22.579 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:22.581 INFO:teuthology.orchestra.run.smithi002.stdout:4847 write 0x4b9462e thru 0x4b9a995 (0x6368 bytes) 2024-09-22T01:02:22.590 INFO:teuthology.orchestra.run.smithi002.stdout:4849 write 0x58bf8bf thru 0x58c95c6 (0x9d08 bytes) 2024-09-22T01:02:22.598 INFO:teuthology.orchestra.run.smithi002.stdout:4852 trunc from 0x58c95c7 to 0x9659819 2024-09-22T01:02:22.603 INFO:teuthology.orchestra.run.smithi002.stdout:4853 read 0x324f404 thru 0x325c4f4 (0xd0f1 bytes) 2024-09-22T01:02:22.615 INFO:teuthology.orchestra.run.smithi002.stdout:4854 write 0xcbd66f1 thru 0xcbe09b4 (0xa2c4 bytes) 2024-09-22T01:02:22.619 INFO:teuthology.orchestra.run.smithi002.stdout:4856 read 0x9b39abd thru 0x9b45662 (0xbba6 bytes) 2024-09-22T01:02:22.619 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.619 INFO:teuthology.orchestra.run.smithi002.stdout:4858 read 0xc815d71 thru 0xc82114d (0xb3dd bytes) 2024-09-22T01:02:22.619 INFO:teuthology.orchestra.run.smithi002.stdout:4859 write 0x5d93768 thru 0x5d98270 (0x4b09 bytes) 2024-09-22T01:02:22.621 INFO:teuthology.orchestra.run.smithi002.stdout:4860 write 0xa6a127e thru 0xa6a548d (0x4210 bytes) 2024-09-22T01:02:22.623 INFO:teuthology.orchestra.run.smithi002.stdout:4862 punch from 0xcabb309 to 0xcac004e, (0x4d45 bytes) 2024-09-22T01:02:22.623 INFO:teuthology.orchestra.run.smithi002.stdout:4863 punch from 0xbb1ce24 to 0xbb1d063, (0x23f bytes) 2024-09-22T01:02:22.623 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.623 INFO:teuthology.orchestra.run.smithi002.stdout:4866 read 0x4538580 thru 0x45406a0 (0x8121 bytes) 2024-09-22T01:02:22.625 INFO:teuthology.orchestra.run.smithi002.stdout:4867 trunc from 0xcbe09b5 to 0xb3a8d8e 2024-09-22T01:02:22.647 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:22.647 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:22.650 INFO:teuthology.orchestra.run.smithi002.stdout:4869 write 0xc0bd24f thru 0xc0c0f87 (0x3d39 bytes) 2024-09-22T01:02:22.655 INFO:teuthology.orchestra.run.smithi002.stdout:4870 write 0xb88292c thru 0xb88ef46 (0xc61b bytes) 2024-09-22T01:02:22.661 INFO:teuthology.orchestra.run.smithi002.stdout:4871 write 0x1672e83 thru 0x167462d (0x17ab bytes) 2024-09-22T01:02:22.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:22.663 INFO:teuthology.orchestra.run.smithi002.stdout:4872 punch from 0x6597f07 to 0x65a6e7e, (0xef77 bytes) 2024-09-22T01:02:22.664 INFO:teuthology.orchestra.run.smithi002.stdout:4874 write 0x346490f thru 0x346dafd (0x91ef bytes) 2024-09-22T01:02:22.666 INFO:teuthology.orchestra.run.smithi002.stdout:4875 write 0x2b47ea1 thru 0x2b4c96c (0x4acc bytes) 2024-09-22T01:02:22.669 INFO:teuthology.orchestra.run.smithi002.stdout:4876 write 0x2e531d9 thru 0x2e61876 (0xe69e bytes) 2024-09-22T01:02:22.679 INFO:teuthology.orchestra.run.smithi002.stdout:4878 write 0xc68e5f1 thru 0xc6961af (0x7bbf bytes) 2024-09-22T01:02:22.684 INFO:teuthology.orchestra.run.smithi002.stdout:4879 writesame 0xc1b21d1 thru 0xc1b4f1d data_size 0x1(0x2d4d bytes) 2024-09-22T01:02:22.696 INFO:teuthology.orchestra.run.smithi002.stdout:4881 punch from 0x2552628 to 0x25531c7, (0xb9f bytes) 2024-09-22T01:02:22.696 INFO:teuthology.orchestra.run.smithi002.stdout:4882 punch from 0x6b81aee to 0x6b83ed1, (0x23e3 bytes) 2024-09-22T01:02:22.705 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:22.702+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:22.715 INFO:teuthology.orchestra.run.smithi002.stdout:4883 trunc from 0xc6961b0 to 0x16d8d7d 2024-09-22T01:02:22.741 INFO:teuthology.orchestra.run.smithi002.stdout:4884 punch from 0x6d042 to 0x754fd, (0x84bb bytes) 2024-09-22T01:02:22.776 INFO:teuthology.orchestra.run.smithi002.stdout:4885 writesame 0xcfbae37 thru 0xcfc0d57 data_size 0x31(0x5f21 bytes) 2024-09-22T01:02:22.776 INFO:teuthology.orchestra.run.smithi002.stdout:4886 trunc from 0xcfc0d58 to 0xcf1431d 2024-09-22T01:02:22.780 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:22.774+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:22.789 INFO:teuthology.orchestra.run.smithi002.stdout:4888 writesame 0xb09bc0d thru 0xb0a651e data_size 0x43(0xa912 bytes) 2024-09-22T01:02:22.796 INFO:teuthology.orchestra.run.smithi002.stdout:4889 read 0x2ce89ca thru 0x2cf3f56 (0xb58d bytes) 2024-09-22T01:02:22.797 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.797 INFO:teuthology.orchestra.run.smithi002.stdout:4892 writesame 0x8322cff thru 0x832e8b9 data_size 0x1(0xbbbb bytes) 2024-09-22T01:02:22.832 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473520928999999, 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473483959999999, 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:02:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473455691000002, 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:02:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:02:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473417815000001, 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:02:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473379224999999, 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:02:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:02:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473306897000001, 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473269031000001, 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:02:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473226791999998, 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:02:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:02:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473172407, 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:02:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.473071642000001, 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:02:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.472984716999999, 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:02:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.472926393000002, 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:02:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:02:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.472473243, 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.471834757, 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.47154716, 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:02:22.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:02:22.895 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.470944191000001, 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:02:22.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.469863891999999, 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:22.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:22.904 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.469794014000001, 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:02:22.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:02:22.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.469232023, 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.912 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.468854598, 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:02:22.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:22.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:22.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:22.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:02:22.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:02:22.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.917 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:22.917 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:22.917 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:22.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4893 read 0xb3b3fd thru 0xb3b552 (0x156 bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4894 read 0x3c69836 thru 0x3c69de5 (0x5b0 bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4895 flatten 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4896 write 0xccf45aa thru 0xccf6744 (0x219b bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4897 write 0x927d63b thru 0x9282bb5 (0x557b bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4898 read 0x2e2187c thru 0x2e2ea7c (0xd201 bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4899 read 0x1640d71 thru 0x164cf29 (0xc1b9 bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4902 write 0xd888ea6 thru 0xd896897 (0xd9f2 bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4903 punch from 0x1807ec5 to 0x180f7e7, (0x7922 bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:4906 write 0x87d5ddd thru 0x87e1f85 (0xc1a9 bytes) 2024-09-22T01:02:22.920 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:22.921 INFO:teuthology.orchestra.run.smithi002.stdout:4908 writesame 0xc5b999e thru 0xc5be011 data_size 0x12(0x4674 bytes) 2024-09-22T01:02:22.921 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:22.917+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:marked out osd.1. 2024-09-22T01:02:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:02:22.930 INFO:teuthology.orchestra.run.smithi002.stdout:4909 punch from 0x584dd31 to 0x5852ee6, (0x51b5 bytes) 2024-09-22T01:02:22.930 INFO:teuthology.orchestra.run.smithi002.stdout:4910 write 0x6d4009d thru 0x6d49d1f (0x9c83 bytes) 2024-09-22T01:02:23.019 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:23.017+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:23.093 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:23.093 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:23.093 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:23.093 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:23.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:23.120 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:23.117+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:23.221 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:23.218+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:23.231 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:23.232 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:23.232 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:23.232 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:23.232 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:23.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:23.337 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:23.334+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 111.521987056, 2024-09-22T01:02:23.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:02:23.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:02:23.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:02:23.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137989728999997, 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:23.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137956424000002, 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:02:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:02:23.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137859911, 2024-09-22T01:02:23.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:02:23.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137786660000003, 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:02:23.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137721479, 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.437 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137651357999999, 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:23.444 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137617992999999, 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:23.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:02:23.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137587570999997, 2024-09-22T01:02:23.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137557264999998, 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:02:23.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:02:23.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137526862999998, 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.456 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137496063999997, 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:02:23.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:02:23.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137465591000002, 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:23.467 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137434999, 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137397196999999, 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.474 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137364832000003, 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.475 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:23.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:02:23.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137334512999999, 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:02:23.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137304620999998, 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.483 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:23.486 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137274861999998, 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:02:23.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.137224396999997, 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:23.490 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:23.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:23.494 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:23.494 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:23.495 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:23.495 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:23.493+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:23.596 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:23.590+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:23.668 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:23.668 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:23.668 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:23.668 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:23.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:23.697 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:23.693+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:23.797 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:23.793+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:23.852 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:23.852 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:23.852 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:23.852 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:23.852 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:23.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:23.913 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:23.910+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866658925999999, 2024-09-22T01:02:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:02:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:02:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:02:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:02:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866603251000001, 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:02:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:02:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866550906000001, 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866492995000002, 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:02:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:02:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866434355999999, 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:02:24.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.061 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866379123999998, 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:24.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:02:24.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866325400000001, 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.067 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:24.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:02:24.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:02:24.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866253665000002, 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:02:24.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:02:24.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:02:24.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.078 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866196283999997, 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.079 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:02:24.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:02:24.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.084 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866146018000002, 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:02:24.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:02:24.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.866066961000001, 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:02:24.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.865942711999999, 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:02:24.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.865862129999996, 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:02:24.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:02:24.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:02:24.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:02:24.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:02:24.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.865796942000003, 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:24.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:02:24.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:02:24.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:02:24.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:02:24.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:02:24.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.117 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.117 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:24.117 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.865021972999998, 2024-09-22T01:02:24.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:02:24.117 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:02:24.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.119 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:24.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:24.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:02:24.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:24.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:02:24.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:02:24.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:02:24.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:02:24.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:02:24.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:02:24.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:02:24.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.864972188000003, 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:24.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:02:24.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:02:24.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:02:24.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:02:24.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:02:24.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:02:24.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:02:24.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:02:24.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:02:24.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.132 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.132 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.133 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:24.133 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.864891086999997, 2024-09-22T01:02:24.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:02:24.133 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:24.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.864821102000001, 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.139 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:02:24.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.864743738000001, 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:02:24.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:02:24.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:02:24.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:02:24.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:02:24.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:02:24.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.151 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.864683217, 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:24.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:24.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:02:24.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:02:24.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:24.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:02:24.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:02:24.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:02:24.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:02:24.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.158 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:24.158 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:24.158 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:24.158 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:23.994+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:24.159 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:24.162 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:24.161+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:24.261 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:24.258+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:24.296 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:24.296 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:24.296 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:24.296 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:24.311 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:24.363 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:24.362+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:24.435 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:24.435 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:24.449 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:24.494 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:24.493+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:24.565 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:24.562+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:24.575 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.075436447000001, 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.576 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:02:24.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:02:24.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.075408604000003, 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:02:24.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.075376388000002, 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.075354650000001, 2024-09-22T01:02:24.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.075334994999999, 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:24.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:02:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:02:24.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.075303980999998, 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:24.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:02:24.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:02:24.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92263 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 20971520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119944813, 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049994333000000002, 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92263 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454537+0000", 2024-09-22T01:02:24.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454541+0000", 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7129999999999999e-06 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454551+0000", 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0448000000000001e-05 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454558+0000", 2024-09-22T01:02:24.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.323e-06 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504317+0000", 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049758335000000001 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504408+0000", 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1508999999999994e-05 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504530+0000", 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122168 2024-09-22T01:02:24.604 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.11989977, 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:02:24.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:02:24.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119860135, 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.609 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119772694, 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:02:24.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:02:24.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119732474999999, 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:02:24.617 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:24.621 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119696583, 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:02:24.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.625 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119648935000001, 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:24.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119614166, 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:02:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:02:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:02:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119580973, 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119536759000001, 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119494570000001, 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:02:24.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:02:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119460458000001, 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119427530999999, 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:02:24.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.654 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.119392763, 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.655 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:02:24.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:02:24.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:24.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:24.659 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:24.659 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:24.660 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:24.707 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:24.705+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:24.767 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:24.762+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:24.831 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:24.832 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:24.848 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:24.898 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:24.897+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:24.958 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:02:24.986 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:24.986 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:24.986 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:24.986 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:24.986 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:24.986 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:24.985+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:25.007 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:25.021 INFO:teuthology.orchestra.run.smithi002.stdout:4912 read 0xa76cbf thru 0xa841f0 (0xd532 bytes) 2024-09-22T01:02:25.023 INFO:teuthology.orchestra.run.smithi002.stdout:4914 punch from 0xb374a99 to 0xb37d80d, (0x8d74 bytes) 2024-09-22T01:02:25.023 INFO:teuthology.orchestra.run.smithi002.stdout:4915 write 0x1d4f0fe thru 0x1d5e78b (0xf68e bytes) 2024-09-22T01:02:25.035 INFO:teuthology.orchestra.run.smithi002.stdout:4916 writesame 0xdd9d422 thru 0xdda841c data_size 0x1(0xaffb bytes) 2024-09-22T01:02:25.038 INFO:teuthology.orchestra.run.smithi002.stdout:4917 writesame 0x4bffe8e thru 0x4c05e9c data_size 0x7(0x600f bytes) 2024-09-22T01:02:25.054 INFO:teuthology.orchestra.run.smithi002.stdout:4919 trunc from 0xdda841d to 0x6b1c416 2024-09-22T01:02:25.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:25.066 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:25.068 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:25.065+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:25.080 INFO:teuthology.orchestra.run.smithi002.stdout:4920 read 0xb5be59 thru 0xb5eec6 (0x306e bytes) 2024-09-22T01:02:25.081 INFO:teuthology.orchestra.run.smithi002.stdout:4921 read 0x781e5f thru 0x789365 (0x7507 bytes) 2024-09-22T01:02:25.091 INFO:teuthology.orchestra.run.smithi002.stdout:4922 writesame 0x6b98a74 thru 0x6b9cc5b data_size 0x4c(0x41e8 bytes) 2024-09-22T01:02:25.094 INFO:teuthology.orchestra.run.smithi002.stdout:4923 writesame 0x1fdbe94 thru 0x1fe4f0b data_size 0x45(0x9078 bytes) 2024-09-22T01:02:25.096 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:25.096 INFO:teuthology.orchestra.run.smithi002.stdout:4925 punch from 0x48e536b to 0x48eae07, (0x5a9c bytes) 2024-09-22T01:02:25.096 INFO:teuthology.orchestra.run.smithi002.stdout:4926 punch from 0x56b926d to 0x56bf8b8, (0x664b bytes) 2024-09-22T01:02:25.097 INFO:teuthology.orchestra.run.smithi002.stdout:4927 clone 43 order 25 su 1048576 sc 8 2024-09-22T01:02:25.097 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:25.097 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:25.125 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:25.126 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:25.151 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:25.151 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:25.169 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:25.165+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:25.176 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:25.177 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:25.192 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:25.192 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:25.203 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:25.203 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:25.215 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:25.237 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:25.238 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:25.270 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:25.266+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:25.270 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:25.271 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:25.371 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:25.369+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:25.379 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:25.379 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:25.394 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:25.472 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:25.466+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:25.548 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:25.548 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:25.561 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:25.610 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:25.609+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:25.673 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:25.670+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:25.676 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:25.676 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:25.690 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:25.774 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:25.770+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:25.848 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:25.848 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:25.863 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:25.910 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:25.909+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:02:25.975 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:25.970+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:25.990 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:25.990 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:26.005 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:26.076 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:26.070+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:26.161 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:26.161 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:26.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:26.177 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:26.173+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:26.279 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:26.277+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:26.306 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:26.306 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:26.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:26.380 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:26.374+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:26.442 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:26.443 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:26.456 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:26.480 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:26.477+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:26.581 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:26.578+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:26.621 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:26.621 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:26.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:26.682 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:26.681+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:26.783 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:26.778+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:26.800 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:26.800 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:26.814 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:26.884 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:26.881+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:26.975 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:26.976 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:26.986 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:26.985+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:26.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:27.086 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:27.085+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:27.118 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:27.118 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:27.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:27.187 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:27.185+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:27.257 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-22T01:02:27.257 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:27.257 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:27.271 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:27.295 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:27.317 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:27.314+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:27.386 INFO:teuthology.orchestra.run.smithi002.stdout:4928 read 0x66ccec1 thru 0x66db52e (0xe66e bytes) 2024-09-22T01:02:27.388 INFO:teuthology.orchestra.run.smithi002.stdout:4932 read 0x55f74d7 thru 0x55fb60e (0x4138 bytes) 2024-09-22T01:02:27.388 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:27.385+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:27.394 INFO:teuthology.orchestra.run.smithi002.stdout:4935 trunc from 0x6b9cc5c to 0x405191e 2024-09-22T01:02:27.395 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:27.395 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:27.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:27.456 INFO:teuthology.orchestra.run.smithi002.stdout:4936 writesame 0xb46577b thru 0xb46a1e2 data_size 0x4(0x4a68 bytes) 2024-09-22T01:02:27.458 INFO:teuthology.orchestra.run.smithi002.stdout:4939 punch from 0x3d0b692 to 0x3d15116, (0x9a84 bytes) 2024-09-22T01:02:27.458 INFO:teuthology.orchestra.run.smithi002.stdout:4941 read 0x5b0cd7b thru 0x5b18ab1 (0xbd37 bytes) 2024-09-22T01:02:27.459 INFO:teuthology.orchestra.run.smithi002.stdout:4942 write 0x6944ea3 thru 0x69531c0 (0xe31e bytes) 2024-09-22T01:02:27.460 INFO:teuthology.orchestra.run.smithi002.stdout:4943 read 0x8039fcf thru 0x803e1bb (0x41ed bytes) 2024-09-22T01:02:27.465 INFO:teuthology.orchestra.run.smithi002.stdout:4944 trunc from 0xb46a1e3 to 0xdba1dab 2024-09-22T01:02:27.469 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:27.469 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:27.471 INFO:teuthology.orchestra.run.smithi002.stdout:4945 read 0xdb4fc5c thru 0xdb54ca0 (0x5045 bytes) 2024-09-22T01:02:27.471 INFO:teuthology.orchestra.run.smithi002.stdout:4946 read 0xd14e76a thru 0xd154760 (0x5ff7 bytes) 2024-09-22T01:02:27.472 INFO:teuthology.orchestra.run.smithi002.stdout:4947 write 0xb5d8c32 thru 0xb5dc92b (0x3cfa bytes) 2024-09-22T01:02:27.474 INFO:teuthology.orchestra.run.smithi002.stdout:4948 read 0x5f32ec4 thru 0x5f3e9d7 (0xbb14 bytes) 2024-09-22T01:02:27.474 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.474 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.474 INFO:teuthology.orchestra.run.smithi002.stdout:4951 write 0x8c0ebe thru 0x8c5462 (0x45a5 bytes) 2024-09-22T01:02:27.475 INFO:teuthology.orchestra.run.smithi002.stdout:4952 read 0x78ca47f thru 0x78ce6b4 (0x4236 bytes) 2024-09-22T01:02:27.476 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.476 INFO:teuthology.orchestra.run.smithi002.stdout:4957 write 0x5688b3 thru 0x57674b (0xde99 bytes) 2024-09-22T01:02:27.478 INFO:teuthology.orchestra.run.smithi002.stdout:4958 read 0x51fb512 thru 0x52069ba (0xb4a9 bytes) 2024-09-22T01:02:27.478 INFO:teuthology.orchestra.run.smithi002.stdout:4959 read 0x32cfb4c thru 0x32d326a (0x371f bytes) 2024-09-22T01:02:27.478 INFO:teuthology.orchestra.run.smithi002.stdout:4960 writesame 0x2e653cf thru 0x2e71ae4 data_size 0x11(0xc716 bytes) 2024-09-22T01:02:27.482 INFO:teuthology.orchestra.run.smithi002.stdout:4961 write 0xd365dcf thru 0xd370ba6 (0xadd8 bytes) 2024-09-22T01:02:27.484 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.484 INFO:teuthology.orchestra.run.smithi002.stdout:4963 write 0xb102ab4 thru 0xb10d01d (0xa56a bytes) 2024-09-22T01:02:27.486 INFO:teuthology.orchestra.run.smithi002.stdout:4964 punch from 0x6a05b0a to 0x6a0b0ac, (0x55a2 bytes) 2024-09-22T01:02:27.486 INFO:teuthology.orchestra.run.smithi002.stdout:4965 write 0x3a4841 thru 0x3ad566 (0x8d26 bytes) 2024-09-22T01:02:27.487 INFO:teuthology.orchestra.run.smithi002.stdout:4966 writesame 0x228698b thru 0x228f6be data_size 0xe(0x8d34 bytes) 2024-09-22T01:02:27.488 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:27.488 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:27.489 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:27.485+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:27.503 INFO:teuthology.orchestra.run.smithi002.stdout:4968 trunc from 0xdba1dab to 0x4d04c86 2024-09-22T01:02:27.507 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:27.507 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:27.526 INFO:teuthology.orchestra.run.smithi002.stdout:4970 write 0x8f4d6ca thru 0x8f5329f (0x5bd6 bytes) 2024-09-22T01:02:27.526 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:27.528 INFO:teuthology.orchestra.run.smithi002.stdout:4972 read 0x25739da thru 0x257f627 (0xbc4e bytes) 2024-09-22T01:02:27.535 INFO:teuthology.orchestra.run.smithi002.stdout:4973 trunc from 0x8f532a0 to 0xd6c3218 2024-09-22T01:02:27.541 INFO:teuthology.orchestra.run.smithi002.stdout:4974 writesame 0x1e37576 thru 0x1e3c44a data_size 0x1f(0x4ed5 bytes) 2024-09-22T01:02:27.544 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.547 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:27.547 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:27.559 INFO:teuthology.orchestra.run.smithi002.stdout:4976 trunc from 0xd6c3218 to 0x2e870e8 2024-09-22T01:02:27.567 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:27.567 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:27.579 INFO:teuthology.orchestra.run.smithi002.stdout:4977 writesame 0x5f4235a thru 0x5f4d0c9 data_size 0x30(0xad70 bytes) 2024-09-22T01:02:27.581 INFO:teuthology.orchestra.run.smithi002.stdout:4978 punch from 0x1dda011 to 0x1de9163, (0xf152 bytes) 2024-09-22T01:02:27.581 INFO:teuthology.orchestra.run.smithi002.stdout:4979 punch from 0x364a1a5 to 0x36509c4, (0x681f bytes) 2024-09-22T01:02:27.581 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.581 INFO:teuthology.orchestra.run.smithi002.stdout:4986 write 0x2b42858 thru 0x2b48ec4 (0x666d bytes) 2024-09-22T01:02:27.581 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:27.581 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:27.590 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:27.590 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:27.586+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:27.590 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:27.591 INFO:teuthology.orchestra.run.smithi002.stdout:4987 trunc from 0x5f4d0ca to 0xb420974 2024-09-22T01:02:27.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:27.600 INFO:teuthology.orchestra.run.smithi002.stdout:4988 trunc from 0xb420974 to 0x9c3abd0 2024-09-22T01:02:27.609 INFO:teuthology.orchestra.run.smithi002.stdout:4989 write 0x54b596f thru 0x54bbba9 (0x623b bytes) 2024-09-22T01:02:27.610 INFO:teuthology.orchestra.run.smithi002.stdout:4991 write 0xc06435 thru 0xc11675 (0xb241 bytes) 2024-09-22T01:02:27.612 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.612 INFO:teuthology.orchestra.run.smithi002.stdout:4995 punch from 0x817c75e to 0x818af63, (0xe805 bytes) 2024-09-22T01:02:27.613 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:27.613 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:27.621 INFO:teuthology.orchestra.run.smithi002.stdout:4997 write 0xbecd5c8 thru 0xbed49ea (0x7423 bytes) 2024-09-22T01:02:27.624 INFO:teuthology.orchestra.run.smithi002.stdout:5000 read 0x5dc6768 thru 0x5dd3378 (0xcc11 bytes) 2024-09-22T01:02:27.625 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.625 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.625 INFO:teuthology.orchestra.run.smithi002.stdout:5003 writesame 0xbe6b3da thru 0xbe7514b data_size 0x7(0x9d72 bytes) 2024-09-22T01:02:27.630 INFO:teuthology.orchestra.run.smithi002.stdout:5004 read 0xaf214d6 thru 0xaf2fbee (0xe719 bytes) 2024-09-22T01:02:27.630 INFO:teuthology.orchestra.run.smithi002.stdout:5005 writesame 0x8ad92ca thru 0x8ad964c data_size 0x1(0x383 bytes) 2024-09-22T01:02:27.632 INFO:teuthology.orchestra.run.smithi002.stdout:5006 read 0x550c126 thru 0x551229b (0x6176 bytes) 2024-09-22T01:02:27.632 INFO:teuthology.orchestra.run.smithi002.stdout:5007 write 0x86322f9 thru 0x86358e2 (0x35ea bytes) 2024-09-22T01:02:27.634 INFO:teuthology.orchestra.run.smithi002.stdout:5009 writesame 0x68eeb30 thru 0x68efcd9 data_size 0x22(0x11aa bytes) 2024-09-22T01:02:27.636 INFO:teuthology.orchestra.run.smithi002.stdout:5010 writesame 0xac1e47b thru 0xac2d380 data_size 0x91(0xef06 bytes) 2024-09-22T01:02:27.637 INFO:teuthology.orchestra.run.smithi002.stdout:5011 write 0x2854e99 thru 0x2857b8d (0x2cf5 bytes) 2024-09-22T01:02:27.639 INFO:teuthology.orchestra.run.smithi002.stdout:5012 punch from 0x12a5600 to 0x12a63ad, (0xdad bytes) 2024-09-22T01:02:27.645 INFO:teuthology.orchestra.run.smithi002.stdout:5013 write 0xc2dc4c5 thru 0xc2e553d (0x9079 bytes) 2024-09-22T01:02:27.647 INFO:teuthology.orchestra.run.smithi002.stdout:5014 read 0x9dd82b7 thru 0x9de0308 (0x8052 bytes) 2024-09-22T01:02:27.648 INFO:teuthology.orchestra.run.smithi002.stdout:5015 punch from 0x2f9b868 to 0x2fa7e74, (0xc60c bytes) 2024-09-22T01:02:27.648 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.648 INFO:teuthology.orchestra.run.smithi002.stdout:5017 write 0x20987fa thru 0x209d3e2 (0x4be9 bytes) 2024-09-22T01:02:27.649 INFO:teuthology.orchestra.run.smithi002.stdout:5018 read 0x976507f thru 0x977371e (0xe6a0 bytes) 2024-09-22T01:02:27.660 INFO:teuthology.orchestra.run.smithi002.stdout:5019 writesame 0xed0a50b thru 0xed15edb data_size 0x1(0xb9d1 bytes) 2024-09-22T01:02:27.662 INFO:teuthology.orchestra.run.smithi002.stdout:5020 writesame 0x259a9ca thru 0x259c6a9 data_size 0x4d(0x1ce0 bytes) 2024-09-22T01:02:27.663 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.663 INFO:teuthology.orchestra.run.smithi002.stdout:5024 punch from 0x64f2f85 to 0x64f989b, (0x6916 bytes) 2024-09-22T01:02:27.664 INFO:teuthology.orchestra.run.smithi002.stdout:5025 write 0x650024 thru 0x657ec9 (0x7ea6 bytes) 2024-09-22T01:02:27.666 INFO:teuthology.orchestra.run.smithi002.stdout:5026 punch from 0x687ddee to 0x688bfe2, (0xe1f4 bytes) 2024-09-22T01:02:27.666 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:27.666 INFO:teuthology.orchestra.run.smithi002.stdout:5029 writesame 0xb788f09 thru 0xb796c2d data_size 0x3(0xdd25 bytes) 2024-09-22T01:02:27.690 INFO:teuthology.orchestra.run.smithi002.stdout:5031 trunc from 0xed15edc to 0x860bb67 2024-09-22T01:02:27.690 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:27.686+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:27.712 INFO:teuthology.orchestra.run.smithi002.stdout:5033 write 0xb02ce2f thru 0xb03444d (0x761f bytes) 2024-09-22T01:02:27.715 INFO:teuthology.orchestra.run.smithi002.stdout:5034 write 0x481c015 thru 0x481cc50 (0xc3c bytes) 2024-09-22T01:02:27.717 INFO:teuthology.orchestra.run.smithi002.stdout:5035 read 0x1984239 thru 0x198b9d9 (0x77a1 bytes) 2024-09-22T01:02:27.718 INFO:teuthology.orchestra.run.smithi002.stdout:5036 write 0x8b5b59b thru 0x8b5f15d (0x3bc3 bytes) 2024-09-22T01:02:27.720 INFO:teuthology.orchestra.run.smithi002.stdout:5037 writesame 0x4c1e71b thru 0x4c26e5a data_size 0x20(0x8740 bytes) 2024-09-22T01:02:27.732 INFO:teuthology.orchestra.run.smithi002.stdout:5038 trunc from 0xb03444e to 0x60677d7 2024-09-22T01:02:27.761 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:27.761 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:27.761 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:27.761 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:27.761 INFO:teuthology.orchestra.run.smithi002.stdout:5040 write 0x929fe5d thru 0x92a1e43 (0x1fe7 bytes) 2024-09-22T01:02:27.764 INFO:teuthology.orchestra.run.smithi002.stdout:5042 read 0x39dd55e thru 0x39e006d (0x2b10 bytes) 2024-09-22T01:02:27.765 INFO:teuthology.orchestra.run.smithi002.stdout:5046 read 0x4d3b7e9 thru 0x4d3c241 (0xa59 bytes) 2024-09-22T01:02:27.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:27.781 INFO:teuthology.orchestra.run.smithi002.stdout:5047 write 0xa92953c thru 0xa937214 (0xdcd9 bytes) 2024-09-22T01:02:27.785 INFO:teuthology.orchestra.run.smithi002.stdout:5048 write 0x1f6babc thru 0x1f77741 (0xbc86 bytes) 2024-09-22T01:02:27.806 INFO:teuthology.orchestra.run.smithi002.stdout:5049 write 0xb4e77b6 thru 0xb4ed073 (0x58be bytes) 2024-09-22T01:02:27.810 INFO:teuthology.orchestra.run.smithi002.stdout:5052 trunc from 0xb4ed074 to 0xaa138dc 2024-09-22T01:02:27.818 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:27.814+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:27.831 INFO:teuthology.orchestra.run.smithi002.stdout:5056 trunc from 0xaa138dc to 0x96e2e5c 2024-09-22T01:02:27.849 INFO:teuthology.orchestra.run.smithi002.stdout:5058 punch from 0x7f64b03 to 0x7f6c411, (0x790e bytes) 2024-09-22T01:02:27.849 INFO:teuthology.orchestra.run.smithi002.stdout:5059 writesame 0x73a0189 thru 0x73a9a7e data_size 0x7(0x98f6 bytes) 2024-09-22T01:02:27.854 INFO:teuthology.orchestra.run.smithi002.stdout:5060 write 0x1f1a4a6 thru 0x1f21985 (0x74e0 bytes) 2024-09-22T01:02:27.874 INFO:teuthology.orchestra.run.smithi002.stdout:5061 trunc from 0x96e2e5c to 0x30632b8 2024-09-22T01:02:27.891 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:27.886+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:27.892 INFO:teuthology.orchestra.run.smithi002.stdout:5063 write 0xffaa66 thru 0xffd062 (0x25fd bytes) 2024-09-22T01:02:27.894 INFO:teuthology.orchestra.run.smithi002.stdout:5065 clone 44 order 19 su 2048 sc 7 2024-09-22T01:02:27.944 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:27.945 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:27.958 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [4, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:02:27.958 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:02:27.959 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:02:27.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:27.993 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:02:28.006 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:28.005+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:28.084 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone43 from 0x30632b8 (overlap 0x2e870e8) to 0x1beaa24 2024-09-22T01:02:28.093 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:28.089+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:28.138 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:28.138 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.006111277999999, 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.139 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:02:28.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:02:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.006055603, 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.144 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.148 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.006003258, 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:02:28.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:02:28.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:02:28.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005945347000001, 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:02:28.153 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:02:28.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005886707999998, 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:02:28.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005831475999997, 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:02:28.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005777752, 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.168 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:02:28.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:02:28.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:02:28.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.172 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005706017000001, 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:28.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:02:28.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:02:28.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:02:28.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005648635999997, 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:02:28.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:02:28.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005598370000001, 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:02:28.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.186 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005519313000001, 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:02:28.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:02:28.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:02:28.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:02:28.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005395063999998, 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:02:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:02:28.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.196 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005314482000003, 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:02:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:02:28.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.005249294000002, 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.201 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:28.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.004474324999997, 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:28.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:02:28.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:02:28.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:02:28.210 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.004424540000002, 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:02:28.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:02:28.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.004343439000003, 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:02:28.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:02:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.004273454, 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:02:28.220 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:02:28.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:02:28.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:02:28.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.004196090000001, 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.004135568999999, 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.232 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:02:28.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:02:28.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:28.236 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:28.237 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-22T01:02:28.238 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:28.234+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:28.238 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:28.297 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:28.298 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:02:28.319 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-22T01:02:27.990772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1373883037,"num_objects":1544,"num_object_clones":435,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":18,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":584,"num_read":141274,"num_read_kb":551164,"num_write":17603,"num_write_kb":309166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1773,"num_bytes_recovered":103407212,"num_keys_recovered":489,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13906,"ondisk_log_size":13906,"up":82,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":533744,"kb_used_data":328876,"kb_used_omap":12,"kb_used_meta":204723,"kb_avail":565697296,"statfs":{"total":579820584960,"available":579274031104,"internally_reserved":0,"allocated":336769024,"data_stored":422532614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12497,"internal_metadata":209637167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":188630200,"num_objects":41,"num_object_clones":14,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":18,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":8,"num_read":2288,"num_read_kb":22220,"num_write":683,"num_write_kb":17615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":42974455,"num_keys_recovered":244,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.064584"},"pg_stats":[{"pgid":"3.1f","version":"98'342","reported_seq":2808,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.939318+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:02:24.939318+0000","last_peered":"2024-09-22T01:02:24.939318+0000","last_clean":"2024-09-22T01:02:24.939318+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:02:24.939318+0000","last_undegraded":"2024-09-22T01:02:24.939318+0000","last_fullsized":"2024-09-22T01:02:24.939318+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:01:42.632136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":61912890,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":14,"num_read":2506,"num_read_kb":31601,"num_write":392,"num_write_kb":12835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"97'620","reported_seq":2786,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.936584+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:02:24.936584+0000","last_peered":"2024-09-22T01:02:24.936584+0000","last_clean":"2024-09-22T01:02:24.936584+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:02:24.936584+0000","last_undegraded":"2024-09-22T01:02:24.936584+0000","last_fullsized":"2024-09-22T01:02:24.936584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:36:57.067364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":28358594,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":5604,"num_read_kb":11515,"num_write":870,"num_write_kb":6629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"102'376","reported_seq":4229,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.975968+0000","last_change":"2024-09-22T01:01:43.138890+0000","last_active":"2024-09-22T01:02:25.975968+0000","last_peered":"2024-09-22T01:02:25.975968+0000","last_clean":"2024-09-22T01:02:25.975968+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:02:25.975968+0000","last_undegraded":"2024-09-22T01:02:25.975968+0000","last_fullsized":"2024-09-22T01:02:25.975968+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:12.597785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81562527,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":4572,"num_read_kb":27937,"num_write":427,"num_write_kb":16397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"102'870","reported_seq":4203,"reported_epoch":102,"state":"peering","last_fresh":"2024-09-22T01:02:25.263344+0000","last_change":"2024-09-22T01:02:25.977035+0000","last_active":"2024-09-22T01:02:25.977035+0000","last_peered":"2024-09-22T01:02:25.263344+0000","last_clean":"2024-09-22T01:02:22.911977+0000","last_became_active":"2024-09-22T01:02:25.005483+0000","last_became_peered":"2024-09-22T01:02:25.005483+0000","last_unstale":"2024-09-22T01:02:25.263344+0000","last_undegraded":"2024-09-22T01:02:25.263344+0000","last_fullsized":"2024-09-22T01:02:24.934292+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":870,"log_dups_size":0,"ondisk_log_size":870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75517897,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":19,"num_read":5819,"num_read_kb":36329,"num_write":1331,"num_write_kb":14583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":18126683,"num_keys_recovered":178,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"98'352","reported_seq":1112,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.939372+0000","last_change":"2024-09-22T01:01:43.147132+0000","last_active":"2024-09-22T01:02:24.939372+0000","last_peered":"2024-09-22T01:02:24.939372+0000","last_clean":"2024-09-22T01:02:24.939372+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:02:24.939372+0000","last_undegraded":"2024-09-22T01:02:24.939372+0000","last_fullsized":"2024-09-22T01:02:24.939372+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:42:53.972976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":26902326,"num_objects":46,"num_object_clones":17,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":1254,"num_read_kb":7145,"num_write":431,"num_write_kb":2153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"99'296","reported_seq":3108,"reported_epoch":102,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:02:25.165345+0000","last_change":"2024-09-22T01:02:25.165345+0000","last_active":"2024-09-22T01:02:25.165345+0000","last_peered":"2024-09-22T01:02:25.165345+0000","last_clean":"2024-09-22T01:02:22.010648+0000","last_became_active":"2024-09-22T01:02:24.947526+0000","last_became_peered":"2024-09-22T01:02:24.947526+0000","last_unstale":"2024-09-22T01:02:25.165345+0000","last_undegraded":"2024-09-22T01:02:25.165345+0000","last_fullsized":"2024-09-22T01:02:24.929235+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":47406279,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":2761,"num_read_kb":6684,"num_write":402,"num_write_kb":7518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2845990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":44}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"95'293","reported_seq":4430,"reported_epoch":102,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:02:25.421758+0000","last_change":"2024-09-22T01:02:25.421758+0000","last_active":"2024-09-22T01:02:25.421758+0000","last_peered":"2024-09-22T01:02:25.421758+0000","last_clean":"2024-09-22T01:02:18.719790+0000","last_became_active":"2024-09-22T01:02:24.962186+0000","last_became_peered":"2024-09-22T01:02:24.962186+0000","last_unstale":"2024-09-22T01:02:25.421758+0000","last_undegraded":"2024-09-22T01:02:25.421758+0000","last_fullsized":"2024-09-22T01:02:24.933751+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":29869828,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":11,"num_read":4167,"num_read_kb":19218,"num_write":364,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2692500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3],"avail_no_missing":["3","1","6"],"object_location_counts":[{"shards":"3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"97'594","reported_seq":5579,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.937574+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:02:24.937574+0000","last_peered":"2024-09-22T01:02:24.937574+0000","last_clean":"2024-09-22T01:02:24.937574+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:02:24.937574+0000","last_undegraded":"2024-09-22T01:02:24.937574+0000","last_fullsized":"2024-09-22T01:02:24.937574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:13.065019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":76931284,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":14,"num_read":7769,"num_read_kb":32984,"num_write":713,"num_write_kb":9038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"99'419","reported_seq":3454,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.939427+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:02:24.939427+0000","last_peered":"2024-09-22T01:02:24.939427+0000","last_clean":"2024-09-22T01:02:24.939427+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:02:24.939427+0000","last_undegraded":"2024-09-22T01:02:24.939427+0000","last_fullsized":"2024-09-22T01:02:24.939427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:41.656227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":35140464,"num_objects":50,"num_object_clones":18,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":4275,"num_read_kb":14548,"num_write":545,"num_write_kb":7017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"99'270","reported_seq":1419,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.976856+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:02:25.976856+0000","last_peered":"2024-09-22T01:02:25.976856+0000","last_clean":"2024-09-22T01:02:25.976856+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:02:25.976856+0000","last_undegraded":"2024-09-22T01:02:25.976856+0000","last_fullsized":"2024-09-22T01:02:25.976856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:12:41.214097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75334913,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":14,"num_read":2233,"num_read_kb":24535,"num_write":339,"num_write_kb":22511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"99'507","reported_seq":5777,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.938262+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:02:24.938262+0000","last_peered":"2024-09-22T01:02:24.938262+0000","last_clean":"2024-09-22T01:02:24.938262+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:02:24.938262+0000","last_undegraded":"2024-09-22T01:02:24.938262+0000","last_fullsized":"2024-09-22T01:02:24.938262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:34:16.919504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":29926438,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":7931,"num_read_kb":23336,"num_write":568,"num_write_kb":2042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"99'397","reported_seq":8984,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.938302+0000","last_change":"2024-09-22T01:01:43.139393+0000","last_active":"2024-09-22T01:02:24.938302+0000","last_peered":"2024-09-22T01:02:24.938302+0000","last_clean":"2024-09-22T01:02:24.938302+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:02:24.938302+0000","last_undegraded":"2024-09-22T01:02:24.938302+0000","last_fullsized":"2024-09-22T01:02:24.938302+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:02.277568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":55132296,"num_objects":50,"num_object_clones":17,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":9297,"num_read_kb":26399,"num_write":466,"num_write_kb":12078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":183,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.973253+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:02:25.973253+0000","last_peered":"2024-09-22T01:02:25.973253+0000","last_clean":"2024-09-22T01:02:25.973253+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:02:25.973253+0000","last_undegraded":"2024-09-22T01:02:25.973253+0000","last_fullsized":"2024-09-22T01:02:25.973253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"102'395","reported_seq":2061,"reported_epoch":102,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:02:25.390937+0000","last_change":"2024-09-22T01:02:25.391197+0000","last_active":"2024-09-22T01:02:25.390937+0000","last_peered":"2024-09-22T01:02:25.390937+0000","last_clean":"2024-09-22T01:02:22.797170+0000","last_became_active":"2024-09-22T01:02:24.970737+0000","last_became_peered":"2024-09-22T01:02:24.970737+0000","last_unstale":"2024-09-22T01:02:25.390937+0000","last_undegraded":"2024-09-22T01:02:24.935213+0000","last_fullsized":"2024-09-22T01:02:24.933803+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":16578833,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":3482,"num_read_kb":4714,"num_write":435,"num_write_kb":1678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":624791,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":["0","1","7"],"object_location_counts":[{"shards":"0","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.973122+0000","last_change":"2024-09-22T01:02:23.944566+0000","last_active":"2024-09-22T01:02:25.973122+0000","last_peered":"2024-09-22T01:02:25.973122+0000","last_clean":"2024-09-22T01:02:25.973122+0000","last_became_active":"2024-09-22T01:02:23.944252+0000","last_became_peered":"2024-09-22T01:02:23.944252+0000","last_unstale":"2024-09-22T01:02:25.973122+0000","last_undegraded":"2024-09-22T01:02:25.973122+0000","last_fullsized":"2024-09-22T01:02:25.973122+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:33.749930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"102'463","reported_seq":2657,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.097781+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:02:25.097781+0000","last_peered":"2024-09-22T01:02:25.097781+0000","last_clean":"2024-09-22T01:02:25.097781+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:02:25.097781+0000","last_undegraded":"2024-09-22T01:02:25.097781+0000","last_fullsized":"2024-09-22T01:02:25.097781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:08.773407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":34093525,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":3470,"num_read_kb":10172,"num_write":622,"num_write_kb":5701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.938733+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:02:24.938733+0000","last_peered":"2024-09-22T01:02:24.938733+0000","last_clean":"2024-09-22T01:02:24.938733+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:02:24.938733+0000","last_undegraded":"2024-09-22T01:02:24.938733+0000","last_fullsized":"2024-09-22T01:02:24.938733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"102'296","reported_seq":3402,"reported_epoch":102,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:02:25.361838+0000","last_change":"2024-09-22T01:02:25.361838+0000","last_active":"2024-09-22T01:02:25.361838+0000","last_peered":"2024-09-22T01:02:25.361838+0000","last_clean":"2024-09-22T01:02:22.623871+0000","last_became_active":"2024-09-22T01:02:24.975142+0000","last_became_peered":"2024-09-22T01:02:24.975142+0000","last_unstale":"2024-09-22T01:02:25.361838+0000","last_undegraded":"2024-09-22T01:02:25.361838+0000","last_fullsized":"2024-09-22T01:02:24.930902+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":34030763,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":3485,"num_read_kb":20439,"num_write":330,"num_write_kb":11250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":5850635,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2],"avail_no_missing":["2","1","5"],"object_location_counts":[{"shards":"2","objects":56}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.939098+0000","last_change":"2024-09-22T01:02:23.944986+0000","last_active":"2024-09-22T01:02:24.939098+0000","last_peered":"2024-09-22T01:02:24.939098+0000","last_clean":"2024-09-22T01:02:24.939098+0000","last_became_active":"2024-09-22T01:02:23.944148+0000","last_became_peered":"2024-09-22T01:02:23.944148+0000","last_unstale":"2024-09-22T01:02:24.939098+0000","last_undegraded":"2024-09-22T01:02:24.939098+0000","last_fullsized":"2024-09-22T01:02:24.939098+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:14:36.383354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"99'484","reported_seq":3535,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.976063+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:02:25.976063+0000","last_peered":"2024-09-22T01:02:25.976063+0000","last_clean":"2024-09-22T01:02:25.976063+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:02:25.976063+0000","last_undegraded":"2024-09-22T01:02:25.976063+0000","last_fullsized":"2024-09-22T01:02:25.976063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:10:08.266425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52812132,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":16,"num_read":5992,"num_read_kb":10814,"num_write":452,"num_write_kb":3992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"102'317","reported_seq":1107,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.071751+0000","last_change":"2024-09-22T01:01:43.148315+0000","last_active":"2024-09-22T01:02:25.071751+0000","last_peered":"2024-09-22T01:02:25.071751+0000","last_clean":"2024-09-22T01:02:25.071751+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:02:25.071751+0000","last_undegraded":"2024-09-22T01:02:25.071751+0000","last_fullsized":"2024-09-22T01:02:25.071751+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:12:20.889330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":50840725,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":20,"num_read":1210,"num_read_kb":6522,"num_write":389,"num_write_kb":11061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":147,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.975796+0000","last_change":"2024-09-22T01:02:23.934997+0000","last_active":"2024-09-22T01:02:25.975796+0000","last_peered":"2024-09-22T01:02:25.975796+0000","last_clean":"2024-09-22T01:02:25.975796+0000","last_became_active":"2024-09-22T01:02:23.931801+0000","last_became_peered":"2024-09-22T01:02:23.931801+0000","last_unstale":"2024-09-22T01:02:25.975796+0000","last_undegraded":"2024-09-22T01:02:25.975796+0000","last_fullsized":"2024-09-22T01:02:25.975796+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:58.752146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"102'1495","reported_seq":3015,"reported_epoch":102,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T01:02:25.669697+0000","last_change":"2024-09-22T01:02:25.422230+0000","last_active":"2024-09-22T01:02:25.669697+0000","last_peered":"2024-09-22T01:02:25.669697+0000","last_clean":"2024-09-22T01:02:22.734235+0000","last_became_active":"2024-09-22T01:02:25.018022+0000","last_became_peered":"2024-09-22T01:02:25.018022+0000","last_unstale":"2024-09-22T01:02:25.669697+0000","last_undegraded":"2024-09-22T01:02:24.949918+0000","last_fullsized":"2024-09-22T01:02:24.929569+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1495,"log_dups_size":0,"ondisk_log_size":1495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":41763088,"num_objects":52,"num_object_clones":17,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":18,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":19,"num_read":8634,"num_read_kb":19497,"num_write":2593,"num_write_kb":10872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3830706,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":18},{"shards":"2","objects":33}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.929185+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:02:24.929185+0000","last_peered":"2024-09-22T01:02:24.929185+0000","last_clean":"2024-09-22T01:02:24.929185+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:02:24.929185+0000","last_undegraded":"2024-09-22T01:02:24.929185+0000","last_fullsized":"2024-09-22T01:02:24.929185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"95'305","reported_seq":2305,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.939571+0000","last_change":"2024-09-22T01:01:43.152419+0000","last_active":"2024-09-22T01:02:24.939571+0000","last_peered":"2024-09-22T01:02:24.939571+0000","last_clean":"2024-09-22T01:02:24.939571+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:02:24.939571+0000","last_undegraded":"2024-09-22T01:02:24.939571+0000","last_fullsized":"2024-09-22T01:02:24.939571+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:44:09.149077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":42829827,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":19,"num_read":2051,"num_read_kb":16935,"num_write":341,"num_write_kb":13327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":186,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.938694+0000","last_change":"2024-09-22T01:02:23.945365+0000","last_active":"2024-09-22T01:02:24.938694+0000","last_peered":"2024-09-22T01:02:24.938694+0000","last_clean":"2024-09-22T01:02:24.938694+0000","last_became_active":"2024-09-22T01:02:23.944743+0000","last_became_peered":"2024-09-22T01:02:23.944743+0000","last_unstale":"2024-09-22T01:02:24.938694+0000","last_undegraded":"2024-09-22T01:02:24.938694+0000","last_fullsized":"2024-09-22T01:02:24.938694+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:32:32.688609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"102'285","reported_seq":1734,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.060220+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:02:25.060220+0000","last_peered":"2024-09-22T01:02:25.060220+0000","last_clean":"2024-09-22T01:02:25.060220+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:02:25.060220+0000","last_undegraded":"2024-09-22T01:02:25.060220+0000","last_fullsized":"2024-09-22T01:02:25.060220+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:33:11.884074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":35328003,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":1496,"num_read_kb":14451,"num_write":306,"num_write_kb":16215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":137,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.975708+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:02:25.975708+0000","last_peered":"2024-09-22T01:02:25.975708+0000","last_clean":"2024-09-22T01:02:25.975708+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:02:25.975708+0000","last_undegraded":"2024-09-22T01:02:25.975708+0000","last_fullsized":"2024-09-22T01:02:25.975708+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.938524+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:02:24.938524+0000","last_peered":"2024-09-22T01:02:24.938524+0000","last_clean":"2024-09-22T01:02:24.938524+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:02:24.938524+0000","last_undegraded":"2024-09-22T01:02:24.938524+0000","last_fullsized":"2024-09-22T01:02:24.938524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'319","reported_seq":9496,"reported_epoch":102,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:02:25.220565+0000","last_change":"2024-09-22T01:02:25.220785+0000","last_active":"2024-09-22T01:02:25.220565+0000","last_peered":"2024-09-22T01:02:25.220565+0000","last_clean":"2024-09-22T01:02:22.861640+0000","last_became_active":"2024-09-22T01:02:24.961910+0000","last_became_peered":"2024-09-22T01:02:24.961910+0000","last_unstale":"2024-09-22T01:02:25.220565+0000","last_undegraded":"2024-09-22T01:02:25.068913+0000","last_fullsized":"2024-09-22T01:02:24.933651+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":57886686,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":23,"num_read":10139,"num_read_kb":25214,"num_write":342,"num_write_kb":11651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":8433203,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3],"avail_no_missing":["3","1","7"],"object_location_counts":[{"shards":"3","objects":61}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"102'437","reported_seq":1645,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.068563+0000","last_change":"2024-09-22T01:01:43.142999+0000","last_active":"2024-09-22T01:02:25.068563+0000","last_peered":"2024-09-22T01:02:25.068563+0000","last_clean":"2024-09-22T01:02:25.068563+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:02:25.068563+0000","last_undegraded":"2024-09-22T01:02:25.068563+0000","last_fullsized":"2024-09-22T01:02:25.068563+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:13.218027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":38736100,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":2349,"num_read_kb":5557,"num_write":535,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"102'564","reported_seq":11162,"reported_epoch":102,"state":"peering","last_fresh":"2024-09-22T01:02:25.220076+0000","last_change":"2024-09-22T01:02:25.974034+0000","last_active":"2024-09-22T01:02:25.974034+0000","last_peered":"2024-09-22T01:02:25.220076+0000","last_clean":"2024-09-22T01:02:22.865886+0000","last_became_active":"2024-09-22T01:02:24.965882+0000","last_became_peered":"2024-09-22T01:02:24.965882+0000","last_unstale":"2024-09-22T01:02:25.220076+0000","last_undegraded":"2024-09-22T01:02:25.220076+0000","last_fullsized":"2024-09-22T01:02:24.933944+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":51916094,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":12207,"num_read_kb":27175,"num_write":630,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":5346356,"num_keys_recovered":98,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"95'391","reported_seq":1888,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.936915+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:02:24.936915+0000","last_peered":"2024-09-22T01:02:24.936915+0000","last_clean":"2024-09-22T01:02:24.936915+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:02:24.936915+0000","last_undegraded":"2024-09-22T01:02:24.936915+0000","last_fullsized":"2024-09-22T01:02:24.936915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:54.890036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":19277367,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":3146,"num_read_kb":15695,"num_write":462,"num_write_kb":7452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"102'304","reported_seq":787,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.978685+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:02:25.978685+0000","last_peered":"2024-09-22T01:02:25.978685+0000","last_clean":"2024-09-22T01:02:25.978685+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:02:25.978685+0000","last_undegraded":"2024-09-22T01:02:25.978685+0000","last_fullsized":"2024-09-22T01:02:25.978685+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:52:05.560524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44996614,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":2385,"num_read_kb":19496,"num_write":334,"num_write_kb":8733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"99'604","reported_seq":5241,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.938329+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:02:24.938329+0000","last_peered":"2024-09-22T01:02:24.938329+0000","last_clean":"2024-09-22T01:02:24.938329+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:02:24.938329+0000","last_undegraded":"2024-09-22T01:02:24.938329+0000","last_fullsized":"2024-09-22T01:02:24.938329+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:15:05.553001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":38875061,"num_objects":59,"num_object_clones":19,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":6956,"num_read_kb":28135,"num_write":787,"num_write_kb":16719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"99'352","reported_seq":3912,"reported_epoch":102,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:02:25.363574+0000","last_change":"2024-09-22T01:02:25.363987+0000","last_active":"2024-09-22T01:02:25.363574+0000","last_peered":"2024-09-22T01:02:25.363574+0000","last_clean":"2024-09-22T01:02:22.201648+0000","last_became_active":"2024-09-22T01:02:24.967751+0000","last_became_peered":"2024-09-22T01:02:24.967751+0000","last_unstale":"2024-09-22T01:02:25.363574+0000","last_undegraded":"2024-09-22T01:02:25.363574+0000","last_fullsized":"2024-09-22T01:02:24.937681+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":48155677,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":3498,"num_read_kb":23996,"num_write":430,"num_write_kb":13672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":7777597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7],"avail_no_missing":["7","0","1"],"object_location_counts":[{"shards":"7","objects":50}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"96'233","reported_seq":3660,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.938511+0000","last_change":"2024-09-22T01:01:43.142453+0000","last_active":"2024-09-22T01:02:24.938511+0000","last_peered":"2024-09-22T01:02:24.938511+0000","last_clean":"2024-09-22T01:02:24.938511+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:02:24.938511+0000","last_undegraded":"2024-09-22T01:02:24.938511+0000","last_fullsized":"2024-09-22T01:02:24.938511+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:23:03.321925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":23429777,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3929,"num_read_kb":4219,"num_write":220,"num_write_kb":7563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"99'324","reported_seq":2536,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.974929+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:02:25.974929+0000","last_peered":"2024-09-22T01:02:25.974929+0000","last_clean":"2024-09-22T01:02:25.974929+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:02:25.974929+0000","last_undegraded":"2024-09-22T01:02:25.974929+0000","last_fullsized":"2024-09-22T01:02:25.974929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:21:42.468972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":41619595,"num_objects":57,"num_object_clones":18,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":2758,"num_read_kb":11777,"num_write":307,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"99'359","reported_seq":1347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.938143+0000","last_change":"2024-09-22T01:01:43.139843+0000","last_active":"2024-09-22T01:02:24.938143+0000","last_peered":"2024-09-22T01:02:24.938143+0000","last_clean":"2024-09-22T01:02:24.938143+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:02:24.938143+0000","last_undegraded":"2024-09-22T01:02:24.938143+0000","last_fullsized":"2024-09-22T01:02:24.938143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:01:20.011778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":32837483,"num_objects":44,"num_object_clones":17,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":17,"num_read":1753,"num_read_kb":6648,"num_write":471,"num_write_kb":4281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"102'315","reported_seq":2467,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:25.076861+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:02:25.076861+0000","last_peered":"2024-09-22T01:02:25.076861+0000","last_clean":"2024-09-22T01:02:25.076861+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:02:25.076861+0000","last_undegraded":"2024-09-22T01:02:25.076861+0000","last_fullsized":"2024-09-22T01:02:25.076861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:18:32.684929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":24020861,"num_objects":44,"num_object_clones":15,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":3092,"num_read_kb":6123,"num_write":329,"num_write_kb":6774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"97'208","reported_seq":1437,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-22T01:02:24.936748+0000","last_change":"2024-09-22T01:02:06.801024+0000","last_active":"2024-09-22T01:02:24.936748+0000","last_peered":"2024-09-22T01:02:24.936748+0000","last_clean":"2024-09-22T01:02:24.936748+0000","last_became_active":"2024-09-22T01:02:06.799165+0000","last_became_peered":"2024-09-22T01:02:06.799165+0000","last_unstale":"2024-09-22T01:02:24.936748+0000","last_undegraded":"2024-09-22T01:02:24.936748+0000","last_fullsized":"2024-09-22T01:02:24.936748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:26:54.116985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":19268703,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":938,"num_read_kb":11254,"num_write":279,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":11755308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1373292650,"num_objects":1540,"num_object_clones":435,"num_object_copies":3080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":18,"num_objects_unfound":0,"num_objects_dirty":1540,"num_whiteouts":584,"num_read":141158,"num_read_kb":551064,"num_write":17442,"num_write_kb":306748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":103407193,"num_keys_recovered":489,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":414490624,"data_stored":523872925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":13786,"ondisk_log_size":13786,"up":64,"acting":57,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509467,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99196,"kb_used_data":59692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94272644,"statfs":{"total":96636764160,"available":96535187456,"internally_reserved":0,"allocated":61124608,"data_stored":77682311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299405,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117172,"kb_used_data":76576,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94254668,"statfs":{"total":96636764160,"available":96516780032,"internally_reserved":0,"allocated":78413824,"data_stored":106545342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":41545294},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71940,"kb_used_data":44908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94299900,"statfs":{"total":96636764160,"available":96563097600,"internally_reserved":0,"allocated":45985792,"data_stored":57804532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542171,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542170,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98568,"kb_used_data":63216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94273272,"statfs":{"total":96636764160,"available":96535830528,"internally_reserved":0,"allocated":64733184,"data_stored":78684329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50008,"kb_used_data":22980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94321832,"statfs":{"total":96636764160,"available":96585555968,"internally_reserved":0,"allocated":23531520,"data_stored":30131484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":60129542172,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96860,"kb_used_data":61504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94274980,"statfs":{"total":96636764160,"available":96537579520,"internally_reserved":0,"allocated":62980096,"data_stored":71684616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61186048,"data_stored":70532945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45142016,"data_stored":58221890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22327296,"data_stored":29570145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":68554752,"data_stored":85003688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":77205504,"data_stored":105983984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":59916288,"data_stored":77120953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53366784,"data_stored":66503323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:02:28.339 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:28.337+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:28.362 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:28.362 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:28.377 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:28.433 INFO:teuthology.orchestra.run.smithi002.stdout:5066 write 0xa513a17 thru 0xa5209db (0xcfc5 bytes) 2024-09-22T01:02:28.439 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:28.434+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:28.459 INFO:teuthology.orchestra.run.smithi002.stdout:5067 write 0x70a457c thru 0x70ae5f1 (0xa076 bytes) 2024-09-22T01:02:28.484 INFO:teuthology.orchestra.run.smithi002.stdout:5068 write 0x7cf6fc8 thru 0x7d00640 (0x9679 bytes) 2024-09-22T01:02:28.506 INFO:teuthology.orchestra.run.smithi002.stdout:5069 clone 45 order 21 su 65536 sc 5 2024-09-22T01:02:28.540 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:28.534+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:28.540 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:28.540 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:28.540 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:28.541 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:28.541 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:28.555 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:28.641 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:28.638+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:28.726 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:28.726 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:28.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:28.726 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:28.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.726 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:28.726 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:28.726 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.224428517, 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:28.730 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.224400674000002, 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:02:28.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:02:28.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.224368458000001, 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:02:28.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:02:28.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:02:28.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.22434672, 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:02:28.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:02:28.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.224327064999997, 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:28.747 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.224296051000003, 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:02:28.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:02:28.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92263 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 20971520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268936882999999, 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049994333000000002, 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92263 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454537+0000", 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454541+0000", 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7129999999999999e-06 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454551+0000", 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0448000000000001e-05 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454558+0000", 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.323e-06 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504317+0000", 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049758335000000001 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504408+0000", 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1508999999999994e-05 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504530+0000", 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122168 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268891839999998, 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.756 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:02:28.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:02:28.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:02:28.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268852205000002, 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:02:28.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268764764, 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:02:28.764 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:02:28.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:02:28.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268724545000001, 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268688653000002, 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:02:28.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268641004999999, 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:28.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:02:28.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268606236, 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:02:28.781 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:02:28.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:02:28.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268573043, 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:02:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:02:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268528829000001, 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268486639999999, 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:02:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268452528000001, 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:02:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:02:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268419601000002, 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:02:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.268384832999999, 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:02:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:28.811 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:28.812 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:28.812 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:28.809+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:28.913 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:28.909+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:28.985 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:28.986 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:29.003 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:29.004 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:02:29.014 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:29.010+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:29.115 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:29.114+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:29.123 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:29.123 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:29.123 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:29.124 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:29.124 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:29.138 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:29.216 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:29.210+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:29.272 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:29.272 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:29.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:29.272 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:29.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.272 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190784777000005, 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190765917999997, 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:02:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.280 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190747732999995, 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:02:29.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:02:29.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:02:29.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190729945000001, 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190711386999993, 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:02:29.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190693335000006, 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.293 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:02:29.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190675986000002, 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:02:29.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:02:29.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190654033000001, 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.301 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:02:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190636498000003, 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:02:29.305 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190612712000004, 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:29.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:02:29.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:02:29.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.313 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190594849999997, 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:02:29.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:02:29.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190567612999999, 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:02:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190548907999997, 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:02:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:02:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190525410999996, 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:02:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:02:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190501441999999, 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.190482227000004, 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:02:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:02:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.189477676999999, 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:02:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.187002409000002, 2024-09-22T01:02:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:02:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.186971724000003, 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:02:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:02:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4715135640000003, 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:02:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:02:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.356 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:29.357 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:29.357 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone44 from 0xa5209dc (overlap 0x30632b8) to 0x0 2024-09-22T01:02:29.358 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:29.354+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:29.458 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:29.457+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:29.559 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:29.554+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107404 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522498+0000", 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046716540000000001, 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046737242999999998, 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107404 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522498+0000", 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522498+0000", 2024-09-22T01:02:29.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522499+0000", 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522502+0000", 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2179999999999999e-06 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522510+0000", 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5650000000000002e-06 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522517+0000", 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0820000000000004e-06 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107474 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.596 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523454+0000", 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045760303000000002, 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045850714000000001, 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107474 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523454+0000", 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523454+0000", 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523455+0000", 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523458+0000", 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5260000000000002e-06 2024-09-22T01:02:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523470+0000", 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1618e-05 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523478+0000", 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7009999999999996e-06 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.599 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107515 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524340+0000", 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044875106999999997, 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045027861000000002, 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107515 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524340+0000", 2024-09-22T01:02:29.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524340+0000", 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524340+0000", 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524342+0000", 2024-09-22T01:02:29.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2589999999999999e-06 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524349+0000", 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524354+0000", 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7690000000000004e-06 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107582 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.531042+0000", 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038172435999999997, 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038386517000000002, 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107582 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.603 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531042+0000", 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531042+0000", 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531043+0000", 2024-09-22T01:02:29.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9999999999999998e-07 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531045+0000", 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.694e-06 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531053+0000", 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9410000000000007e-06 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531058+0000", 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1370000000000001e-06 2024-09-22T01:02:29.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107662 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538991+0000", 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030223678, 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030498150000000002, 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107662 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538991+0000", 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538991+0000", 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538991+0000", 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538994+0000", 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4279999999999999e-06 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539000+0000", 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6059999999999999e-06 2024-09-22T01:02:29.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539006+0000", 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2399999999999998e-06 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108167 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546819+0000", 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022396027999999998, 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022734438999999999, 2024-09-22T01:02:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108167 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546819+0000", 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546819+0000", 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546819+0000", 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1899999999999999e-07 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546821+0000", 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6240000000000002e-06 2024-09-22T01:02:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546827+0000", 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546832+0000", 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3140000000000003e-06 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108198 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554816+0000", 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014398344, 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014797319999999999, 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108198 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.613 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554816+0000", 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554816+0000", 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554817+0000", 2024-09-22T01:02:29.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8299999999999998e-07 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554819+0000", 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.418e-06 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554826+0000", 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5490000000000003e-06 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554831+0000", 2024-09-22T01:02:29.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0220000000000004e-06 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108349 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.562840+0000", 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0063744409999999998, 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068352209999999998, 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108349 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562840+0000", 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562840+0000", 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562841+0000", 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562843+0000", 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8270000000000002e-06 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562851+0000", 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2259999999999999e-06 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562856+0000", 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4049999999999999e-06 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107407 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522527+0000", 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046687870999999999, 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047211217999999999, 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107407 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522527+0000", 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522527+0000", 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522527+0000", 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522530+0000", 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.379e-06 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522536+0000", 2024-09-22T01:02:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8619999999999997e-06 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522542+0000", 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0900000000000004e-06 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107476 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523494+0000", 2024-09-22T01:02:29.622 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045720711999999997, 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046307663999999998, 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107476 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523494+0000", 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523494+0000", 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523495+0000", 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.61e-07 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523498+0000", 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.954e-06 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523505+0000", 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523511+0000", 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9359999999999996e-06 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107516 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.625 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524361+0000", 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044853759, 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045502009000000003, 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107516 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524361+0000", 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524361+0000", 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524361+0000", 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7800000000000002e-07 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524363+0000", 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-22T01:02:29.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524370+0000", 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3269999999999997e-06 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524375+0000", 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0470000000000001e-06 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107586 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.628 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.531066+0000", 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038148502000000001, 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038860862000000003, 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107586 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531066+0000", 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531066+0000", 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531067+0000", 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0400000000000002e-07 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531069+0000", 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7e-06 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531076+0000", 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5339999999999996e-06 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531081+0000", 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8520000000000003e-06 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108137 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.539030+0000", 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030184493, 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030958602000000002, 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108137 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539030+0000", 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539030+0000", 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539031+0000", 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2799999999999995e-07 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539034+0000", 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3009999999999998e-06 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539043+0000", 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3930000000000008e-06 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539049+0000", 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108168 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546840+0000", 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022374344000000001, 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023211244999999998, 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108168 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546840+0000", 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546840+0000", 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546841+0000", 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546842+0000", 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.677e-06 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546846+0000", 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546850+0000", 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4860000000000002e-06 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108199 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554840+0000", 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014375068, 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015273484, 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108199 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.638 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554840+0000", 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554840+0000", 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554840+0000", 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6800000000000001e-07 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554842+0000", 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4040000000000002e-06 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554849+0000", 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3910000000000003e-06 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554854+0000", 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0409999999999997e-06 2024-09-22T01:02:29.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108350 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.562863+0000", 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0063513320000000003, 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073103760000000004, 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108350 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.641 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562863+0000", 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562863+0000", 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562864+0000", 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562866+0000", 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562873+0000", 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9410000000000001e-06 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562878+0000", 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.31e-06 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.643 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107409 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522549+0000", 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046665921999999999, 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047686877000000003, 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107409 2024-09-22T01:02:29.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522549+0000", 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522549+0000", 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522549+0000", 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4900000000000001e-07 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522551+0000", 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2670000000000001e-06 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522558+0000", 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.776e-06 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522563+0000", 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107477 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523519+0000", 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045695721000000002, 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046782149000000002, 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107477 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523519+0000", 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523519+0000", 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523519+0000", 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523522+0000", 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.339e-06 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523528+0000", 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6150000000000002e-06 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523534+0000", 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2310000000000004e-06 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107517 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524440+0000", 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044774639999999997, 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045921115999999998, 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107517 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524440+0000", 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524440+0000", 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524441+0000", 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524444+0000", 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999999e-06 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524453+0000", 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6589999999999996e-06 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524460+0000", 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-06 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107588 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.531087+0000", 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038127540000000001, 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039336614999999998, 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107588 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531087+0000", 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531087+0000", 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531088+0000", 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-07 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531090+0000", 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3240000000000001e-06 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531096+0000", 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1129999999999997e-06 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531101+0000", 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8999999999999997e-06 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108138 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.539058+0000", 2024-09-22T01:02:29.656 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030156481999999998, 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031426933999999997, 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108138 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539058+0000", 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539058+0000", 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539059+0000", 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539061+0000", 2024-09-22T01:02:29.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3539999999999998e-06 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539067+0000", 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9959999999999999e-06 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539072+0000", 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7770000000000002e-06 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108169 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.659 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546854+0000", 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022360976000000001, 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023691924999999999, 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108169 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546854+0000", 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546854+0000", 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546854+0000", 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4299999999999999e-07 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546855+0000", 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546859+0000", 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3340000000000002e-06 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546862+0000", 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.163e-06 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.662 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108200 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554873+0000", 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014341774, 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015749484000000001, 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108200 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554873+0000", 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554873+0000", 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554873+0000", 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554876+0000", 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7760000000000002e-06 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554883+0000", 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1409999999999999e-06 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554888+0000", 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2100000000000001e-06 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108351 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.562885+0000", 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0063292330000000001, 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077765229999999996, 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108351 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562885+0000", 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562885+0000", 2024-09-22T01:02:29.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562886+0000", 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6899999999999998e-07 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562894+0000", 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9149999999999996e-06 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562901+0000", 2024-09-22T01:02:29.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8080000000000005e-06 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562909+0000", 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4259999999999997e-06 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107411 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522579+0000", 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046635652, 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048116386999999997, 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107411 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522579+0000", 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522579+0000", 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522579+0000", 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5200000000000002e-07 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522582+0000", 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6620000000000001e-06 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522589+0000", 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8940000000000004e-06 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522594+0000", 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7609999999999998e-06 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107479 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523541+0000", 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045673915000000002, 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047208704999999997, 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107479 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523541+0000", 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523541+0000", 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523541+0000", 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.96e-07 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523543+0000", 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.108e-06 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523550+0000", 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523554+0000", 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5709999999999999e-06 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107518 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524470+0000", 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044744151000000003, 2024-09-22T01:02:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046339572000000002, 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107518 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524470+0000", 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524470+0000", 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524471+0000", 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.08e-07 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524474+0000", 2024-09-22T01:02:29.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.632e-06 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524481+0000", 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9959999999999996e-06 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524486+0000", 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1139999999999997e-06 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107590 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.531120+0000", 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038094331000000002, 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039754601000000001, 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107590 2024-09-22T01:02:29.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.681 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531120+0000", 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531120+0000", 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535937+0000", 2024-09-22T01:02:29.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048170510000000001 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535949+0000", 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1289e-05 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535961+0000", 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2023e-05 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535969+0000", 2024-09-22T01:02:29.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0039999999999999e-06 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108139 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.539079+0000", 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030135769999999999, 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031855896000000002, 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.684 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108139 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539079+0000", 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539079+0000", 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539079+0000", 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.15e-07 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539082+0000", 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3870000000000002e-06 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539088+0000", 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1709999999999999e-06 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539092+0000", 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4750000000000004e-06 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108170 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546890+0000", 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022324911999999999, 2024-09-22T01:02:29.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024086666, 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108170 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546890+0000", 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546890+0000", 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546890+0000", 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6499999999999999e-07 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546894+0000", 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4570000000000001e-06 2024-09-22T01:02:29.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546904+0000", 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0411000000000001e-05 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546912+0000", 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4430000000000003e-06 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108201 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554896+0000", 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014318177, 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016136343000000001, 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108201 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554896+0000", 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554896+0000", 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554897+0000", 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.89e-07 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554899+0000", 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4279999999999999e-06 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554906+0000", 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5030000000000002e-06 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554911+0000", 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4659999999999998e-06 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108352 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.562963+0000", 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0062519669999999998, 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081194630000000004, 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108352 2024-09-22T01:02:29.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562963+0000", 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562963+0000", 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562963+0000", 2024-09-22T01:02:29.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7100000000000001e-07 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562967+0000", 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8539999999999999e-06 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562977+0000", 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.024e-05 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562985+0000", 2024-09-22T01:02:29.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1099999999999997e-06 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107413 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522607+0000", 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046608068000000002, 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048523539999999997, 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107413 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522607+0000", 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522607+0000", 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522607+0000", 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522610+0000", 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6730000000000001e-06 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522617+0000", 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.148e-06 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522621+0000", 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6720000000000003e-06 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107481 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523561+0000", 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045653517999999997, 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047618875999999997, 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107481 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523561+0000", 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523561+0000", 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523562+0000", 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523564+0000", 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0509999999999999e-06 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523569+0000", 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.75e-06 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523574+0000", 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1710000000000004e-06 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107519 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524493+0000", 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044721625000000001, 2024-09-22T01:02:29.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046791903000000003, 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107519 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524493+0000", 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524493+0000", 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524493+0000", 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.84e-07 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524496+0000", 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3190000000000002e-06 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524504+0000", 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6079999999999996e-06 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524510+0000", 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2839999999999997e-06 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.706 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107592 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.535983+0000", 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033231229000000001, 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035357685, 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107592 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535983+0000", 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535983+0000", 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535984+0000", 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.06e-07 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535986+0000", 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4650000000000001e-06 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.535999+0000", 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2607e-05 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536016+0000", 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6901e-05 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108140 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.539099+0000", 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030115782000000001, 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032304976999999999, 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108140 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.710 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539099+0000", 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539099+0000", 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539099+0000", 2024-09-22T01:02:29.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539101+0000", 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.939e-06 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539107+0000", 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5779999999999998e-06 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539111+0000", 2024-09-22T01:02:29.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2139999999999998e-06 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108202 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554919+0000", 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014296125999999999, 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016536781, 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108202 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554919+0000", 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554919+0000", 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554919+0000", 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554921+0000", 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.289e-06 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554928+0000", 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6449999999999999e-06 2024-09-22T01:02:29.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554935+0000", 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7959999999999998e-06 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108353 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.562995+0000", 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0062195410000000003, 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0085200810000000005, 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108353 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562995+0000", 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562995+0000", 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562996+0000", 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5200000000000002e-07 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562998+0000", 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8899999999999999e-06 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563006+0000", 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2450000000000001e-06 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563012+0000", 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107415 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522630+0000", 2024-09-22T01:02:29.719 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046585093000000001, 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048954312, 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107415 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522630+0000", 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522630+0000", 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522630+0000", 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522639+0000", 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5880000000000004e-06 2024-09-22T01:02:29.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522656+0000", 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7003999999999998e-05 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522662+0000", 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8340000000000004e-06 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107483 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.722 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523580+0000", 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045634647, 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048066518000000003, 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107483 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523580+0000", 2024-09-22T01:02:29.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523580+0000", 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523580+0000", 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523583+0000", 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-22T01:02:29.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523589+0000", 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8409999999999996e-06 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523593+0000", 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0199999999999996e-06 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.725 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107520 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524517+0000", 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044697164999999997, 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047195645000000001, 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107520 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524517+0000", 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524517+0000", 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524617+0000", 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9648000000000005e-05 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524621+0000", 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9920000000000001e-06 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524631+0000", 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9389999999999992e-06 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524638+0000", 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6980000000000001e-06 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107594 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536025+0000", 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033190048999999999, 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035753678999999997, 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107594 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536025+0000", 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536025+0000", 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536025+0000", 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1300000000000001e-07 2024-09-22T01:02:29.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536028+0000", 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5720000000000001e-06 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536034+0000", 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6699999999999997e-06 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536039+0000", 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0679999999999996e-06 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108141 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.539116+0000", 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030098356999999999, 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032726592999999998, 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108141 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539116+0000", 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539116+0000", 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539117+0000", 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8700000000000002e-07 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539119+0000", 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539124+0000", 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2109999999999997e-06 2024-09-22T01:02:29.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539128+0000", 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1919999999999998e-06 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108171 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546923+0000", 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022291681000000001, 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024990637, 2024-09-22T01:02:29.735 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108171 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546923+0000", 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546923+0000", 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550698+0000", 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037753790000000001 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550705+0000", 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9480000000000002e-06 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550719+0000", 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550728+0000", 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8789999999999992e-06 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108203 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554942+0000", 2024-09-22T01:02:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014272284, 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017037968000000001, 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108203 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554942+0000", 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554942+0000", 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554943+0000", 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554945+0000", 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1730000000000002e-06 2024-09-22T01:02:29.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554951+0000", 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4420000000000003e-06 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554956+0000", 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0490000000000003e-06 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.741 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108354 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.563020+0000", 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.006194209, 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0090246760000000006, 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108354 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563020+0000", 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563020+0000", 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563021+0000", 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563024+0000", 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563028+0000", 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9150000000000004e-06 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563032+0000", 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2459999999999999e-06 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107417 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522672+0000", 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046542279999999998, 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049438922000000003, 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107417 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522672+0000", 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522672+0000", 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522673+0000", 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.58e-07 2024-09-22T01:02:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522676+0000", 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.875e-06 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522683+0000", 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7039999999999994e-06 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522688+0000", 2024-09-22T01:02:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107485 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523600+0000", 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045614933000000003, 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048546160999999997, 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107485 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523600+0000", 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523600+0000", 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523600+0000", 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6600000000000002e-07 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523602+0000", 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1909999999999999e-06 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523608+0000", 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.071e-06 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523613+0000", 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6489999999999999e-06 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107521 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524648+0000", 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044566177999999998, 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047530797999999999, 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107521 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524648+0000", 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524648+0000", 2024-09-22T01:02:29.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526671+0000", 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020227270000000002 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526677+0000", 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7429999999999999e-06 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526690+0000", 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2668e-05 2024-09-22T01:02:29.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526697+0000", 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8800000000000008e-06 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107600 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536046+0000", 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033168707999999998, 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036185433000000003, 2024-09-22T01:02:29.754 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107600 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536046+0000", 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536046+0000", 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536046+0000", 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536048+0000", 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8980000000000001e-06 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536054+0000", 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.049e-06 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536059+0000", 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108142 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.757 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.539133+0000", 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030081488999999999, 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033158423999999999, 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108142 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539133+0000", 2024-09-22T01:02:29.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539133+0000", 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539133+0000", 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6399999999999998e-07 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539136+0000", 2024-09-22T01:02:29.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4049999999999998e-06 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539141+0000", 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3630000000000002e-06 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539145+0000", 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0849999999999999e-06 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.760 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108172 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.550778+0000", 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018436923000000001, 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021547581999999999, 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108172 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550778+0000", 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550778+0000", 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550778+0000", 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2900000000000003e-07 2024-09-22T01:02:29.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550788+0000", 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5230000000000001e-06 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550797+0000", 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2080000000000006e-06 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550804+0000", 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1380000000000001e-06 2024-09-22T01:02:29.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108204 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554964+0000", 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014251051000000001, 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017395273999999999, 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108204 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554964+0000", 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554964+0000", 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554964+0000", 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6600000000000002e-07 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554966+0000", 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3369999999999998e-06 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554973+0000", 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.37e-06 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554978+0000", 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9690000000000002e-06 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108355 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.563037+0000", 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.006177976, 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093546480000000001, 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.767 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108355 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563037+0000", 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563037+0000", 2024-09-22T01:02:29.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563037+0000", 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2700000000000001e-07 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563038+0000", 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3379999999999999e-06 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563042+0000", 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-06 2024-09-22T01:02:29.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563047+0000", 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8899999999999998e-06 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107419 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522697+0000", 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046517619000000003, 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049727223000000001, 2024-09-22T01:02:29.770 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107419 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522697+0000", 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522697+0000", 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522697+0000", 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522700+0000", 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5050000000000002e-06 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522706+0000", 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5200000000000003e-06 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522711+0000", 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107487 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523620+0000", 2024-09-22T01:02:29.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045594556000000001, 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048837006000000002, 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107487 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523620+0000", 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523620+0000", 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523620+0000", 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523628+0000", 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7770000000000002e-06 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523635+0000", 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9140000000000002e-06 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523640+0000", 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9019999999999998e-06 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107523 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.526713+0000", 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.042501878, 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045774540000000002, 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107523 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526713+0000", 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526713+0000", 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526714+0000", 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5300000000000003e-07 2024-09-22T01:02:29.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526717+0000", 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2509999999999999e-06 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526725+0000", 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.507e-06 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526731+0000", 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:29.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107602 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536065+0000", 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033149495000000001, 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036472704000000002, 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107602 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536065+0000", 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536065+0000", 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536065+0000", 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9900000000000002e-07 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536067+0000", 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8080000000000001e-06 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536073+0000", 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9270000000000002e-06 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536078+0000", 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108143 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.539151+0000", 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030063998000000001, 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033446851999999999, 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108143 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539151+0000", 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539151+0000", 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539198+0000", 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6891000000000002e-05 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539201+0000", 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6229999999999999e-06 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539209+0000", 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6590000000000007e-06 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539215+0000", 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108173 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.550815+0000", 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018399285000000001, 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021873747999999998, 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108173 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550815+0000", 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550815+0000", 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550816+0000", 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9800000000000004e-07 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550819+0000", 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0960000000000001e-06 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550827+0000", 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7190000000000001e-06 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550832+0000", 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.575e-06 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108205 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554984+0000", 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014230349, 2024-09-22T01:02:29.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017787431999999999, 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108205 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554984+0000", 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554984+0000", 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554985+0000", 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5900000000000003e-07 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554987+0000", 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.283e-06 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554993+0000", 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2890000000000003e-06 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554998+0000", 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4610000000000001e-06 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108356 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.563051+0000", 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0061639080000000001, 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0097611120000000006, 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108356 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563051+0000", 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563051+0000", 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563051+0000", 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.11e-07 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563052+0000", 2024-09-22T01:02:29.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.277e-06 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563056+0000", 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4079999999999998e-06 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563058+0000", 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8739999999999999e-06 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107421 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522719+0000", 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046495704999999998, 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050147035999999999, 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107421 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522719+0000", 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522719+0000", 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522719+0000", 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-07 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522721+0000", 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1600000000000001e-06 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522728+0000", 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6479999999999997e-06 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522733+0000", 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7589999999999997e-06 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107489 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523647+0000", 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045567149000000001, 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049305478999999999, 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107489 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.799 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523647+0000", 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523647+0000", 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523648+0000", 2024-09-22T01:02:29.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6899999999999998e-07 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523651+0000", 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0180000000000002e-06 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523658+0000", 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6619999999999999e-06 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523662+0000", 2024-09-22T01:02:29.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107525 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.526742+0000", 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.042473033, 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046272915999999997, 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107525 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526742+0000", 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526742+0000", 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526742+0000", 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8100000000000002e-07 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526745+0000", 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1159999999999999e-06 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526753+0000", 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.622e-06 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526759+0000", 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8980000000000001e-06 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107604 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536084+0000", 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033130813000000002, 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037005365999999998, 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107604 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536084+0000", 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536084+0000", 2024-09-22T01:02:29.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536084+0000", 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9999999999999999e-07 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536086+0000", 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7850000000000001e-06 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536092+0000", 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8819999999999998e-06 2024-09-22T01:02:29.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536096+0000", 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5569999999999997e-06 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108144 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.539224+0000", 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.029990533, 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033898269000000002, 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108144 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539224+0000", 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.539224+0000", 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542584+0000", 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003359894 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542591+0000", 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9469999999999997e-06 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542605+0000", 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3696000000000001e-05 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542613+0000", 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5839999999999995e-06 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108174 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.550840+0000", 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.01837461, 2024-09-22T01:02:29.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022398405, 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108174 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550840+0000", 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550840+0000", 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550840+0000", 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9499999999999998e-07 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550843+0000", 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.661e-06 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550850+0000", 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8129999999999998e-06 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550855+0000", 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0540000000000002e-06 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108206 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.814 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.555007+0000", 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014207951999999999, 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018318996000000001, 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108206 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.555007+0000", 2024-09-22T01:02:29.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.555007+0000", 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558634+0000", 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036277219999999999 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558640+0000", 2024-09-22T01:02:29.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7679999999999997e-06 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558657+0000", 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6663999999999998e-05 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558665+0000", 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3869999999999995e-06 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108357 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.563062+0000", 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0061524250000000004, 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010329260999999999, 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108357 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563062+0000", 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563062+0000", 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563062+0000", 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1400000000000001e-07 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563064+0000", 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-06 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563067+0000", 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.303e-06 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563070+0000", 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.751e-06 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107428 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522740+0000", 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046474623, 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050713011000000002, 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107428 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522740+0000", 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522740+0000", 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522740+0000", 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522743+0000", 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.198e-06 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522749+0000", 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522754+0000", 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7079999999999996e-06 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.823 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107491 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523669+0000", 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045545430999999997, 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049852678999999997, 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107491 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523669+0000", 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523669+0000", 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523670+0000", 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0600000000000001e-07 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523672+0000", 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1780000000000002e-06 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523678+0000", 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.686e-06 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523683+0000", 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7010000000000004e-06 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107527 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.526767+0000", 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.042447174999999997, 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046791551000000001, 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107527 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526767+0000", 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526767+0000", 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526768+0000", 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526770+0000", 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2670000000000001e-06 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526777+0000", 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0870000000000001e-06 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526782+0000", 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107606 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536106+0000", 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033108911999999997, 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037484851999999999, 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107606 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536106+0000", 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536106+0000", 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536106+0000", 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536109+0000", 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5840000000000002e-06 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536116+0000", 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-06 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536121+0000", 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5570000000000003e-06 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108145 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542658+0000", 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026556682000000002, 2024-09-22T01:02:29.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031014692, 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108145 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542658+0000", 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542658+0000", 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542659+0000", 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8899999999999999e-07 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542662+0000", 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7809999999999999e-06 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542672+0000", 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2199999999999998e-06 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542678+0000", 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8449999999999997e-06 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108175 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.550887+0000", 2024-09-22T01:02:29.836 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018328025000000001, 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022847844999999999, 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108175 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550887+0000", 2024-09-22T01:02:29.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550887+0000", 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550887+0000", 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6599999999999995e-07 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550892+0000", 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5379999999999996e-06 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550901+0000", 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550909+0000", 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5170000000000002e-06 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108207 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.558680+0000", 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0105348, 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015119304, 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108207 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558680+0000", 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558680+0000", 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558680+0000", 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1600000000000001e-07 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558684+0000", 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5719999999999999e-06 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558692+0000", 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1729999999999995e-06 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558698+0000", 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.842 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108358 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.563073+0000", 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.006141535, 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010809762000000001, 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108358 2024-09-22T01:02:29.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563073+0000", 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.563073+0000", 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566786+0000", 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00371318 2024-09-22T01:02:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566792+0000", 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6319999999999996e-06 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566811+0000", 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9043e-05 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566842+0000", 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1377999999999997e-05 2024-09-22T01:02:29.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107430 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522761+0000", 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046453502000000001, 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051207576999999997, 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107430 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522761+0000", 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522761+0000", 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522761+0000", 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522763+0000", 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9810000000000002e-06 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522770+0000", 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3820000000000001e-06 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522774+0000", 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4989999999999996e-06 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107493 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523690+0000", 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045524436000000001, 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050321671999999998, 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107493 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523690+0000", 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523690+0000", 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523691+0000", 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9099999999999999e-07 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523693+0000", 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523699+0000", 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3609999999999997e-06 2024-09-22T01:02:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523704+0000", 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8280000000000002e-06 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107529 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.526790+0000", 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.042424677000000001, 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047257122999999998, 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107529 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526790+0000", 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526790+0000", 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526790+0000", 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526793+0000", 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3779999999999999e-06 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526800+0000", 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.114e-06 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526805+0000", 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107608 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536158+0000", 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033057093000000003, 2024-09-22T01:02:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037919962000000002, 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107608 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536158+0000", 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536158+0000", 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536158+0000", 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-07 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536163+0000", 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1739999999999999e-06 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536170+0000", 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2069999999999998e-06 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536176+0000", 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0610000000000003e-06 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108146 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542689+0000", 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026525433000000001, 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031420667999999999, 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108146 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542689+0000", 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542689+0000", 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542690+0000", 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.58e-07 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542692+0000", 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7080000000000002e-06 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542700+0000", 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8339999999999999e-06 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542706+0000", 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5339999999999999e-06 2024-09-22T01:02:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108176 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.550922+0000", 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018292976999999998, 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023218576000000001, 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108176 2024-09-22T01:02:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550922+0000", 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550922+0000", 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550922+0000", 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6800000000000001e-07 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550925+0000", 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.757e-06 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550932+0000", 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2679999999999997e-06 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550938+0000", 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6869999999999999e-06 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108208 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.558723+0000", 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010491993999999999, 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015475063000000001, 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108208 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558723+0000", 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558723+0000", 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558723+0000", 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1200000000000003e-07 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558731+0000", 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2220000000000002e-06 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558740+0000", 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4780000000000008e-06 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558746+0000", 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1199999999999999e-06 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108359 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.566858+0000", 2024-09-22T01:02:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.002356332, 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074005609999999999, 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108359 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566858+0000", 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566858+0000", 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566859+0000", 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-07 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566863+0000", 2024-09-22T01:02:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6950000000000002e-06 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566872+0000", 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0240000000000003e-06 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566878+0000", 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5769999999999999e-06 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107432 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522781+0000", 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046433373, 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051566381000000001, 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107432 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522781+0000", 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522781+0000", 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522782+0000", 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:02:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522784+0000", 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2050000000000001e-06 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522790+0000", 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.331e-06 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522795+0000", 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.904e-06 2024-09-22T01:02:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107494 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523712+0000", 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045503120000000001, 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050697491999999997, 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107494 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523712+0000", 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523712+0000", 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523712+0000", 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6899999999999998e-07 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523714+0000", 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4260000000000002e-06 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523721+0000", 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7700000000000004e-06 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523726+0000", 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3569999999999997e-06 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107531 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.526812+0000", 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.042402695999999997, 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047658965999999997, 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107531 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526812+0000", 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526812+0000", 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526812+0000", 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7e-07 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526820+0000", 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0290000000000005e-06 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526828+0000", 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8169999999999999e-06 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526834+0000", 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5099999999999998e-06 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107611 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536183+0000", 2024-09-22T01:02:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033031363000000001, 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038375447, 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107611 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536183+0000", 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536183+0000", 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536184+0000", 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536186+0000", 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4090000000000001e-06 2024-09-22T01:02:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536192+0000", 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2949999999999999e-06 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536197+0000", 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5399999999999997e-06 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108147 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542714+0000", 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026500229, 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031912056000000001, 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108147 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542714+0000", 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542714+0000", 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542715+0000", 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542717+0000", 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6290000000000002e-06 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542725+0000", 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3010000000000001e-06 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542730+0000", 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.237e-06 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108177 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.550948+0000", 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018267129, 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023744660000000001, 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108177 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550948+0000", 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550948+0000", 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550948+0000", 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.32e-07 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550950+0000", 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5220000000000002e-06 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550972+0000", 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2011e-05 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550980+0000", 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2640000000000002e-06 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108209 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.558757+0000", 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010458108000000001, 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015967269999999999, 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108209 2024-09-22T01:02:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558757+0000", 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558757+0000", 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558757+0000", 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1600000000000002e-07 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558759+0000", 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558767+0000", 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3370000000000002e-06 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558775+0000", 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9510000000000006e-06 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108360 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.566888+0000", 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.002326795, 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0078697839999999995, 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108360 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566888+0000", 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566888+0000", 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566888+0000", 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566892+0000", 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.399e-06 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566899+0000", 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4159999999999998e-06 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566905+0000", 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6960000000000002e-06 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107434 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522810+0000", 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046404969999999997, 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051978580000000003, 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107434 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522810+0000", 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522810+0000", 2024-09-22T01:02:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522810+0000", 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2300000000000002e-07 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522812+0000", 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3099999999999999e-06 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522819+0000", 2024-09-22T01:02:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0639999999999996e-06 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522825+0000", 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4670000000000003e-06 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107495 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523733+0000", 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045481234000000002, 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051088498000000003, 2024-09-22T01:02:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107495 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523733+0000", 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523733+0000", 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523734+0000", 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523736+0000", 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523742+0000", 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1789999999999996e-06 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523747+0000", 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1529999999999996e-06 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107536 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.526842+0000", 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.042373069999999999, 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048010561, 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107536 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526842+0000", 2024-09-22T01:02:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526842+0000", 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526842+0000", 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.01e-07 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526844+0000", 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5179999999999999e-06 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526851+0000", 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8650000000000003e-06 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526857+0000", 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3210000000000004e-06 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107613 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536204+0000", 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033010169999999998, 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038690382000000002, 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107613 2024-09-22T01:02:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536204+0000", 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536204+0000", 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536205+0000", 2024-09-22T01:02:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.89e-07 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536207+0000", 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2800000000000002e-06 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536214+0000", 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4609999999999996e-06 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536219+0000", 2024-09-22T01:02:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0270000000000003e-06 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108148 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542739+0000", 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026476045, 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032186823000000003, 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108148 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542739+0000", 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542739+0000", 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542739+0000", 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8500000000000002e-07 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542747+0000", 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3739999999999998e-06 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542755+0000", 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6499999999999996e-06 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542761+0000", 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.071e-06 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108178 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.550990+0000", 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.01822495, 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023993535999999999, 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108178 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550990+0000", 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550990+0000", 2024-09-22T01:02:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550990+0000", 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0800000000000005e-07 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.550993+0000", 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8940000000000002e-06 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551000+0000", 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0539999999999997e-06 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551007+0000", 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9589999999999998e-06 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108210 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.558840+0000", 2024-09-22T01:02:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010374454, 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016224182, 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108210 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558840+0000", 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558840+0000", 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558841+0000", 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.68e-07 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558845+0000", 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0550000000000001e-06 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558856+0000", 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0686000000000001e-05 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558866+0000", 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9529999999999994e-06 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108361 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.566914+0000", 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0023009459999999999, 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082167019999999993, 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108361 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566914+0000", 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566914+0000", 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566914+0000", 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566917+0000", 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-06 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566924+0000", 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566929+0000", 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0669999999999999e-06 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107436 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522832+0000", 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046382159999999999, 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052358862999999999, 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107436 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522832+0000", 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522832+0000", 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522833+0000", 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522835+0000", 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3140000000000002e-06 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522842+0000", 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7519999999999999e-06 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522847+0000", 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6260000000000003e-06 2024-09-22T01:02:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107496 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523754+0000", 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045460233000000003, 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051525311999999997, 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107496 2024-09-22T01:02:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523754+0000", 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523754+0000", 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523755+0000", 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523757+0000", 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.435e-06 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523763+0000", 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2489999999999998e-06 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523769+0000", 2024-09-22T01:02:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107538 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.526864+0000", 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.042350763, 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048477363000000002, 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107538 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526864+0000", 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526864+0000", 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526864+0000", 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526867+0000", 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526873+0000", 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.635e-06 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526879+0000", 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6919999999999999e-06 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107615 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536228+0000", 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.03298649, 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039194472000000001, 2024-09-22T01:02:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107615 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536228+0000", 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536228+0000", 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536229+0000", 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3200000000000005e-07 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536232+0000", 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.19e-06 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536241+0000", 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5469999999999997e-06 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536248+0000", 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3180000000000001e-06 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108149 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542770+0000", 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026445059999999999, 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032711288999999998, 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108149 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542770+0000", 2024-09-22T01:02:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542770+0000", 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542770+0000", 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9999999999999998e-07 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542773+0000", 2024-09-22T01:02:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6560000000000001e-06 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542780+0000", 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1080000000000004e-06 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542785+0000", 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3510000000000001e-06 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108179 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551015+0000", 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018199869, 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024528590999999999, 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108179 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551015+0000", 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551015+0000", 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551015+0000", 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.01e-07 2024-09-22T01:02:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551018+0000", 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4480000000000001e-06 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551033+0000", 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4942e-05 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551043+0000", 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0546e-05 2024-09-22T01:02:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108330 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.558878+0000", 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010336491999999999, 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016733458999999999, 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108330 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558878+0000", 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558878+0000", 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558879+0000", 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7800000000000001e-07 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558882+0000", 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2459999999999999e-06 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558890+0000", 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5569999999999998e-06 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558897+0000", 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7069999999999992e-06 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108362 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.566943+0000", 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0022720539999999999, 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0087164210000000002, 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108362 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566943+0000", 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566943+0000", 2024-09-22T01:02:29.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566943+0000", 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.32e-07 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566946+0000", 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8279999999999999e-06 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.566953+0000", 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3329999999999999e-06 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573090+0000", 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061366010000000002 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107438 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522854+0000", 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046360875000000003, 2024-09-22T01:02:29.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052887866999999998, 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107438 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522854+0000", 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522854+0000", 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522854+0000", 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522865+0000", 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1252e-05 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522874+0000", 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0280000000000006e-06 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522879+0000", 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1120000000000004e-06 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.954 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107497 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523776+0000", 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045438523000000001, 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052031767, 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107497 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523776+0000", 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523776+0000", 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523776+0000", 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523779+0000", 2024-09-22T01:02:29.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2290000000000002e-06 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523785+0000", 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4099999999999996e-06 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523790+0000", 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107540 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.526886+0000", 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.042328694, 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048988651000000001, 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107540 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526886+0000", 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.526886+0000", 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530581+0000", 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036955209999999998 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530587+0000", 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4450000000000004e-06 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530601+0000", 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4034e-05 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530610+0000", 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6319999999999997e-06 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107617 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536259+0000", 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032956007000000002, 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039660981999999997, 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107617 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536259+0000", 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536259+0000", 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536259+0000", 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536263+0000", 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.715e-06 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536272+0000", 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.132e-06 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536281+0000", 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.963 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108150 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542793+0000", 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026421601999999999, 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033177632999999998, 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108150 2024-09-22T01:02:29.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542793+0000", 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542793+0000", 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542793+0000", 2024-09-22T01:02:29.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.77e-07 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542796+0000", 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542803+0000", 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2629999999999997e-06 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542808+0000", 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2499999999999997e-06 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108180 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551055+0000", 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018159676, 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024947268000000002, 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108180 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551055+0000", 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551055+0000", 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551055+0000", 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7599999999999997e-07 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551059+0000", 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0910000000000001e-06 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551065+0000", 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551075+0000", 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6770000000000007e-06 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108331 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.558906+0000", 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010308533999999999, 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017133031, 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.970 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108331 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558906+0000", 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558906+0000", 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558907+0000", 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7700000000000005e-07 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558909+0000", 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.717e-06 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558916+0000", 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9369999999999998e-06 2024-09-22T01:02:29.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558924+0000", 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3579999999999997e-06 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108363 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.573111+0000", 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9961038, 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029915380000000002, 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108363 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573111+0000", 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573111+0000", 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573112+0000", 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1009999999999999e-06 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573121+0000", 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9039999999999998e-06 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573129+0000", 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0590000000000003e-06 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573135+0000", 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7180000000000001e-06 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107440 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522887+0000", 2024-09-22T01:02:29.976 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046327213999999999, 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053274519999999999, 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107440 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522887+0000", 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522887+0000", 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522888+0000", 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522894+0000", 2024-09-22T01:02:29.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6139999999999997e-06 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522903+0000", 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3049999999999993e-06 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522908+0000", 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999999e-06 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107498 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523797+0000", 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045417467000000003, 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052426245000000003, 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107498 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523797+0000", 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523797+0000", 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523798+0000", 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7e-07 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523800+0000", 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0360000000000001e-06 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523807+0000", 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523813+0000", 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.198e-06 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107542 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530624+0000", 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038590317999999998, 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045664239000000002, 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107542 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.983 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530624+0000", 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530624+0000", 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530625+0000", 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6199999999999998e-07 2024-09-22T01:02:29.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530628+0000", 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4340000000000001e-06 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530637+0000", 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1980000000000001e-06 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530643+0000", 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0469999999999999e-06 2024-09-22T01:02:29.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107619 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536298+0000", 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032916413999999998, 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040050616999999997, 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107619 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536298+0000", 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536298+0000", 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536299+0000", 2024-09-22T01:02:29.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1600000000000002e-07 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536306+0000", 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6529999999999994e-06 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536316+0000", 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4639999999999995e-06 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536322+0000", 2024-09-22T01:02:29.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1689999999999997e-06 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108151 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542816+0000", 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026398413999999999, 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033593431, 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108151 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542816+0000", 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542816+0000", 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542817+0000", 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0999999999999999e-07 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542820+0000", 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0230000000000001e-06 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542827+0000", 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9129999999999997e-06 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542832+0000", 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3680000000000001e-06 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108181 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551109+0000", 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018105658, 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025367099000000001, 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108181 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551109+0000", 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551109+0000", 2024-09-22T01:02:29.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551110+0000", 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3499999999999996e-07 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551113+0000", 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5559999999999999e-06 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551121+0000", 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8080000000000005e-06 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551143+0000", 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2447999999999999e-05 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108332 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.558940+0000", 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010274634, 2024-09-22T01:02:29.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017596193, 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108332 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558940+0000", 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558940+0000", 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558940+0000", 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558943+0000", 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8700000000000001e-06 2024-09-22T01:02:29.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558951+0000", 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558956+0000", 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.631e-06 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108364 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.573141+0000", 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9960737, 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034634919999999999, 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108364 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:29.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573141+0000", 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573141+0000", 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573141+0000", 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573143+0000", 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6640000000000001e-06 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573147+0000", 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2150000000000003e-06 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573150+0000", 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0079999999999998e-06 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107442 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522956+0000", 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046258716999999998, 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053711821999999999, 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107442 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522956+0000", 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522956+0000", 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522956+0000", 2024-09-22T01:02:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522960+0000", 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522976+0000", 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5539000000000001e-05 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522983+0000", 2024-09-22T01:02:30.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3969999999999997e-06 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107499 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523820+0000", 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045395096000000003, 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052917659999999998, 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.005 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107499 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523820+0000", 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523820+0000", 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523820+0000", 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-07 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523822+0000", 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1430000000000001e-06 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523828+0000", 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523833+0000", 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6600000000000003e-06 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107545 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530653+0000", 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038561834000000003, 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046146354000000001, 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107545 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530653+0000", 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530653+0000", 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530653+0000", 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6499999999999999e-07 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530656+0000", 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1159999999999999e-06 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530664+0000", 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.17e-06 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530670+0000", 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107621 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536331+0000", 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032883213000000001, 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040534809999999998, 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107621 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536331+0000", 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536331+0000", 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536332+0000", 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8799999999999998e-07 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536334+0000", 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5330000000000001e-06 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536342+0000", 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.221e-06 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536347+0000", 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0010000000000001e-06 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108152 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542839+0000", 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026375189, 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034092418999999999, 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.014 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108152 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542839+0000", 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542839+0000", 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542840+0000", 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7899999999999999e-07 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542843+0000", 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8080000000000001e-06 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542848+0000", 2024-09-22T01:02:30.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5099999999999998e-06 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542854+0000", 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6790000000000002e-06 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108182 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551180+0000", 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018034963000000001, 2024-09-22T01:02:30.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025817212999999999, 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108182 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551180+0000", 2024-09-22T01:02:30.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551180+0000", 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551180+0000", 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7700000000000004e-07 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551184+0000", 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.489e-06 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551192+0000", 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.875e-06 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551198+0000", 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1999999999999999e-06 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108333 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.558965+0000", 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010249220999999999, 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018092595, 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108333 2024-09-22T01:02:30.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558965+0000", 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558965+0000", 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558966+0000", 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558968+0000", 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6809999999999998e-06 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558975+0000", 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9319999999999999e-06 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.558981+0000", 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108365 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.573154+0000", 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9960604, 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039702970000000002, 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108365 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573154+0000", 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573154+0000", 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573154+0000", 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573156+0000", 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.57e-06 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573160+0000", 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6559999999999998e-06 2024-09-22T01:02:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573162+0000", 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7669999999999999e-06 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107443 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.522993+0000", 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046221305999999997, 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05419305, 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107443 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522993+0000", 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522993+0000", 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522994+0000", 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.522997+0000", 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8150000000000002e-06 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523004+0000", 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5020000000000003e-06 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523012+0000", 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5399999999999998e-06 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107500 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523840+0000", 2024-09-22T01:02:30.030 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045374475999999997, 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053723541, 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107500 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523840+0000", 2024-09-22T01:02:30.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523840+0000", 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523841+0000", 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.84e-07 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523843+0000", 2024-09-22T01:02:30.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3209999999999999e-06 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523849+0000", 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523855+0000", 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107547 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530679+0000", 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038535604000000001, 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046949238999999997, 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107547 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530679+0000", 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530679+0000", 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530679+0000", 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530682+0000", 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6809999999999998e-06 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530689+0000", 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1319999999999996e-06 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530695+0000", 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6049999999999997e-06 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.036 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107623 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536353+0000", 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032861376999999997, 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041342556000000003, 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107623 2024-09-22T01:02:30.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536353+0000", 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536353+0000", 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536354+0000", 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536356+0000", 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536362+0000", 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4620000000000001e-06 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536367+0000", 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0950000000000003e-06 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108153 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542889+0000", 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026325454000000002, 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034839716999999999, 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108153 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542889+0000", 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542889+0000", 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542890+0000", 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542892+0000", 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.768e-06 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542901+0000", 2024-09-22T01:02:30.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2630000000000003e-06 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542907+0000", 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1940000000000003e-06 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108183 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551225+0000", 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017989363000000001, 2024-09-22T01:02:30.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026536917, 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108183 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551225+0000", 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551225+0000", 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551226+0000", 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6100000000000001e-07 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551229+0000", 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.416e-06 2024-09-22T01:02:30.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551243+0000", 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3685e-05 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551250+0000", 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.601e-06 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108334 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560073+0000", 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0091414029999999993, 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017720085999999999, 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108334 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560073+0000", 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560073+0000", 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560074+0000", 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3300000000000002e-07 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560083+0000", 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2320000000000007e-06 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560094+0000", 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1151999999999999e-05 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560102+0000", 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8800000000000008e-06 2024-09-22T01:02:30.049 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108366 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.573166+0000", 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9960485, 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046598789999999996, 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108366 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573166+0000", 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573166+0000", 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573167+0000", 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0100000000000001e-07 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573168+0000", 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.666e-06 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573172+0000", 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4630000000000001e-06 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573174+0000", 2024-09-22T01:02:30.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.571e-06 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107445 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523020+0000", 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046194342999999999, 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054838339, 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107445 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523020+0000", 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523020+0000", 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523021+0000", 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0400000000000002e-07 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523024+0000", 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0230000000000001e-06 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523031+0000", 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8360000000000003e-06 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523036+0000", 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.186e-06 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107501 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523862+0000", 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045352341999999997, 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054045374, 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107501 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523862+0000", 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523862+0000", 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523863+0000", 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523865+0000", 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0490000000000002e-06 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523871+0000", 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3470000000000004e-06 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523876+0000", 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107550 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530703+0000", 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038511239000000003, 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047258175999999999, 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107550 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530703+0000", 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530703+0000", 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530704+0000", 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6499999999999998e-07 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530707+0000", 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530714+0000", 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-22T01:02:30.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530720+0000", 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0340000000000004e-06 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107626 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536378+0000", 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032836669999999998, 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041645045999999998, 2024-09-22T01:02:30.062 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107626 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536378+0000", 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536378+0000", 2024-09-22T01:02:30.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536378+0000", 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536381+0000", 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7209999999999999e-06 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536388+0000", 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3900000000000004e-06 2024-09-22T01:02:30.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536394+0000", 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6969999999999998e-06 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108154 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542916+0000", 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026298800000000001, 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035172773999999997, 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108154 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542916+0000", 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542916+0000", 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542916+0000", 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4400000000000001e-07 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542919+0000", 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3439999999999999e-06 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542925+0000", 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542930+0000", 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0950000000000003e-06 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108184 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551259+0000", 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017955606999999998, 2024-09-22T01:02:30.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026893263000000001, 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108184 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551259+0000", 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551259+0000", 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551259+0000", 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551268+0000", 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5450000000000003e-06 2024-09-22T01:02:30.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551313+0000", 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5037999999999998e-05 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551320+0000", 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108335 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560137+0000", 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0090780300000000008, 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018067873000000002, 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108335 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560137+0000", 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560137+0000", 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560137+0000", 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-07 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560141+0000", 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6030000000000001e-06 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560149+0000", 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5199999999999997e-06 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560156+0000", 2024-09-22T01:02:30.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6749999999999996e-06 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108367 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.573178+0000", 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9960365, 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050586720000000002, 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108367 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573178+0000", 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573178+0000", 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573178+0000", 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1899999999999999e-07 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573180+0000", 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3909999999999999e-06 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573183+0000", 2024-09-22T01:02:30.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2550000000000002e-06 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573186+0000", 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3409999999999999e-06 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107447 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523043+0000", 2024-09-22T01:02:30.078 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046171258, 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055226248999999998, 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107447 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523043+0000", 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523043+0000", 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523044+0000", 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8099999999999998e-07 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523046+0000", 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2280000000000001e-06 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523054+0000", 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5639999999999999e-06 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523059+0000", 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1259999999999997e-06 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107502 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523884+0000", 2024-09-22T01:02:30.081 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045331035999999998, 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054418148999999999, 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107502 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523884+0000", 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523884+0000", 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523884+0000", 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.84e-07 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523892+0000", 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5429999999999996e-06 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523900+0000", 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9430000000000006e-06 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523905+0000", 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107554 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530730+0000", 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038484974999999998, 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047606054000000002, 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107554 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530730+0000", 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530730+0000", 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530730+0000", 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:02:30.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530735+0000", 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6879999999999998e-06 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530742+0000", 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3849999999999996e-06 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530748+0000", 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8150000000000002e-06 2024-09-22T01:02:30.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107628 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536457+0000", 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032757965999999999, 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041911123000000002, 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107628 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.088 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536457+0000", 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536457+0000", 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536457+0000", 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1799999999999995e-07 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536461+0000", 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6660000000000001e-06 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536471+0000", 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0362e-05 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536478+0000", 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:02:30.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108155 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542937+0000", 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026277168, 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035463186000000001, 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108155 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542937+0000", 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542937+0000", 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542938+0000", 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8500000000000002e-07 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542941+0000", 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0130000000000002e-06 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542948+0000", 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0639999999999996e-06 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542953+0000", 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4190000000000001e-06 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108185 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551340+0000", 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017874926999999999, 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027111057000000001, 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108185 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551340+0000", 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551340+0000", 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551340+0000", 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.75e-07 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551344+0000", 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4960000000000001e-06 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551352+0000", 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8779999999999998e-06 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551358+0000", 2024-09-22T01:02:30.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9989999999999997e-06 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108336 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560166+0000", 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0090482140000000006, 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018349074, 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108336 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560166+0000", 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560166+0000", 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560167+0000", 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5499999999999998e-07 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560169+0000", 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5979999999999999e-06 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560177+0000", 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560182+0000", 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3970000000000002e-06 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108368 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.573193+0000", 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9960222, 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053929959999999997, 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108368 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573193+0000", 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573193+0000", 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573193+0000", 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0699999999999998e-07 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573196+0000", 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9510000000000002e-06 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573202+0000", 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573207+0000", 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6120000000000001e-06 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107449 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523066+0000", 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046149021999999998, 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055582529999999998, 2024-09-22T01:02:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107449 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523066+0000", 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523066+0000", 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523066+0000", 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523068+0000", 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0329999999999998e-06 2024-09-22T01:02:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523074+0000", 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1859999999999997e-06 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523079+0000", 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9969999999999998e-06 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107503 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523913+0000", 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045301397, 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054781028000000002, 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107503 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523913+0000", 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523913+0000", 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523914+0000", 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523916+0000", 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523922+0000", 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4110000000000001e-06 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523927+0000", 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8930000000000004e-06 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107557 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530757+0000", 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038457166000000001, 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047968031000000001, 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107557 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530757+0000", 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530757+0000", 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530758+0000", 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0600000000000001e-07 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530760+0000", 2024-09-22T01:02:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6259999999999999e-06 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530768+0000", 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0729999999999999e-06 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530773+0000", 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4020000000000001e-06 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107630 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536489+0000", 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032725855999999998, 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042271972999999997, 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107630 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536489+0000", 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536489+0000", 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536489+0000", 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536492+0000", 2024-09-22T01:02:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7779999999999999e-06 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536500+0000", 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6170000000000001e-06 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536506+0000", 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.071e-06 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108186 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551400+0000", 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017814776000000001, 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.02739219, 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108186 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551400+0000", 2024-09-22T01:02:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551400+0000", 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551400+0000", 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6599999999999996e-07 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551404+0000", 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3340000000000002e-06 2024-09-22T01:02:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551412+0000", 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2470000000000008e-06 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551418+0000", 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2380000000000002e-06 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108337 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560190+0000", 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0090245610000000004, 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018634726000000001, 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108337 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560190+0000", 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560190+0000", 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560191+0000", 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560193+0000", 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5210000000000001e-06 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560199+0000", 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3199999999999996e-06 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560205+0000", 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5550000000000002e-06 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108369 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.573215+0000", 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9959993, 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056410940000000001, 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108369 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573215+0000", 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573215+0000", 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573216+0000", 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.32e-07 2024-09-22T01:02:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573219+0000", 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7810000000000001e-06 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573224+0000", 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573227+0000", 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2160000000000002e-06 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107450 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523086+0000", 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046128251000000002, 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055802577999999999, 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107450 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523086+0000", 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523086+0000", 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523087+0000", 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7e-07 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523089+0000", 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.097e-06 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523095+0000", 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5289999999999997e-06 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523100+0000", 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8210000000000001e-06 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107504 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523941+0000", 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045273745999999997, 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054978481000000003, 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107504 2024-09-22T01:02:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523941+0000", 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523941+0000", 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523941+0000", 2024-09-22T01:02:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0600000000000001e-07 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523944+0000", 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8540000000000001e-06 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524014+0000", 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9617999999999997e-05 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524022+0000", 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4689999999999997e-06 2024-09-22T01:02:30.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107559 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530781+0000", 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038433276000000002, 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048171867, 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107559 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530781+0000", 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530781+0000", 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530782+0000", 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530785+0000", 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.824e-06 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530791+0000", 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6549999999999998e-06 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530796+0000", 2024-09-22T01:02:30.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.011e-06 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107632 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536516+0000", 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032698905, 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042468242000000003, 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107632 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536516+0000", 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536516+0000", 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536516+0000", 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3300000000000003e-07 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536519+0000", 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7020000000000001e-06 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536526+0000", 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3919999999999997e-06 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536532+0000", 2024-09-22T01:02:30.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108156 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.542961+0000", 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.026253812000000001, 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036057921, 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108156 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542961+0000", 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.542961+0000", 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546606+0000", 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036451470000000001 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546612+0000", 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8270000000000003e-06 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546624+0000", 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2418e-05 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546632+0000", 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6410000000000002e-06 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108187 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551428+0000", 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017786712999999999, 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027650053000000001, 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.140 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108187 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551428+0000", 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551428+0000", 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551428+0000", 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4900000000000001e-07 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551431+0000", 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9349999999999999e-06 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551438+0000", 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9550000000000003e-06 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551477+0000", 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9011999999999999e-05 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108338 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560213+0000", 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0090014989999999996, 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018932188999999999, 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108338 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560213+0000", 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560213+0000", 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560214+0000", 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1899999999999998e-07 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560217+0000", 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0929999999999999e-06 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560224+0000", 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9670000000000004e-06 2024-09-22T01:02:30.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560229+0000", 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2530000000000004e-06 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108370 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.573232+0000", 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9959826, 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005974717, 2024-09-22T01:02:30.146 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108370 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573232+0000", 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.573232+0000", 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574660+0000", 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014282940000000001 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574666+0000", 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9309999999999996e-06 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574679+0000", 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2641e-05 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574687+0000", 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1470000000000001e-06 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107451 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523107+0000", 2024-09-22T01:02:30.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046107532999999999, 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056157600000000002, 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107451 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523107+0000", 2024-09-22T01:02:30.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523107+0000", 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523107+0000", 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8599999999999999e-07 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523109+0000", 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9910000000000001e-06 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523116+0000", 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6769999999999998e-06 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523121+0000", 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9760000000000003e-06 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107505 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524034+0000", 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045180755000000003, 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055269873999999997, 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107505 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524034+0000", 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524034+0000", 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524034+0000", 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8999999999999996e-07 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524038+0000", 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.579e-06 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524046+0000", 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2300000000000008e-06 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524053+0000", 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107561 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530804+0000", 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038410148999999998, 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048545115999999999, 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107561 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530804+0000", 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530804+0000", 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530805+0000", 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530807+0000", 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5689999999999999e-06 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530814+0000", 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530819+0000", 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2259999999999996e-06 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107634 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.536542+0000", 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.032673109999999998, 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042842515999999997, 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107634 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536542+0000", 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.536542+0000", 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538714+0000", 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021726029999999999 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538721+0000", 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4679999999999997e-06 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538732+0000", 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1654000000000001e-05 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538740+0000", 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1189999999999997e-06 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108157 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546647+0000", 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022567802000000001, 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032770129000000002, 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108157 2024-09-22T01:02:30.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546647+0000", 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546647+0000", 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546647+0000", 2024-09-22T01:02:30.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3600000000000003e-07 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546651+0000", 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5049999999999999e-06 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546658+0000", 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4359999999999996e-06 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546664+0000", 2024-09-22T01:02:30.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108188 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551491+0000", 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017723543000000001, 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027970654000000001, 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.165 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108188 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551491+0000", 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551491+0000", 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551492+0000", 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551495+0000", 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2940000000000001e-06 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551502+0000", 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1509999999999998e-06 2024-09-22T01:02:30.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551508+0000", 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8690000000000001e-06 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108339 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560237+0000", 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0089777669999999993, 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019255970000000001, 2024-09-22T01:02:30.168 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108339 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560237+0000", 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560237+0000", 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560237+0000", 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0400000000000002e-07 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560268+0000", 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0626999999999997e-05 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560278+0000", 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0091e-05 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560284+0000", 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108371 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.171 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.574701+0000", 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9945135, 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048329089999999998, 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108371 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574701+0000", 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574701+0000", 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574702+0000", 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574704+0000", 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.34e-06 2024-09-22T01:02:30.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574710+0000", 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6319999999999996e-06 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574713+0000", 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.805e-06 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.174 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107453 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523128+0000", 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046086268, 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056451886, 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107453 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523128+0000", 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523128+0000", 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523129+0000", 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.77e-07 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523131+0000", 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4250000000000001e-06 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523146+0000", 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4725999999999999e-05 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523152+0000", 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0220000000000001e-06 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107506 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524061+0000", 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045153760000000001, 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055585230999999999, 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107506 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.178 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524061+0000", 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524061+0000", 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524061+0000", 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1399999999999997e-07 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524064+0000", 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4669999999999998e-06 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524071+0000", 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1420000000000004e-06 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524076+0000", 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0170000000000004e-06 2024-09-22T01:02:30.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107563 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530827+0000", 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.03838743, 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048878735999999999, 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.181 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107563 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530827+0000", 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530827+0000", 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530828+0000", 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530836+0000", 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4139999999999994e-06 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530843+0000", 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8859999999999998e-06 2024-09-22T01:02:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530848+0000", 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.045e-06 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107636 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538755+0000", 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030459235000000001, 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041016866999999999, 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107636 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538755+0000", 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538755+0000", 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538756+0000", 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0800000000000004e-07 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538759+0000", 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0450000000000001e-06 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538767+0000", 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9019999999999999e-06 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538773+0000", 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8170000000000004e-06 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108158 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.187 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546674+0000", 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022540503999999999, 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033141060999999999, 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108158 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546674+0000", 2024-09-22T01:02:30.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546674+0000", 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546675+0000", 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546677+0000", 2024-09-22T01:02:30.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546681+0000", 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3059999999999999e-06 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546684+0000", 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.033e-06 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108189 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551546+0000", 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017668755000000001, 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.02830394, 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108189 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551546+0000", 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551546+0000", 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551546+0000", 2024-09-22T01:02:30.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4300000000000003e-07 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551550+0000", 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4470000000000002e-06 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551558+0000", 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6090000000000001e-06 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551587+0000", 2024-09-22T01:02:30.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8150999999999999e-05 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108340 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560292+0000", 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0089230390000000007, 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019593047999999998, 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108340 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560292+0000", 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560292+0000", 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560292+0000", 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560294+0000", 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0559999999999999e-06 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560298+0000", 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9140000000000001e-06 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560301+0000", 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108372 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.574719+0000", 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9944959, 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005217373, 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108372 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574719+0000", 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574719+0000", 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574719+0000", 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7099999999999998e-07 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574721+0000", 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.468e-06 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574724+0000", 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7510000000000002e-06 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574727+0000", 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8059999999999999e-06 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107455 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523214+0000", 2024-09-22T01:02:30.200 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.046000455000000003, 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056753894999999999, 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107455 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523214+0000", 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523214+0000", 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523215+0000", 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8699999999999995e-07 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523218+0000", 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4479999999999999e-06 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523228+0000", 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0032e-05 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523235+0000", 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.635e-06 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107507 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.203 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524083+0000", 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045131922999999997, 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055919284999999999, 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107507 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524083+0000", 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524083+0000", 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524083+0000", 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524085+0000", 2024-09-22T01:02:30.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0789999999999999e-06 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524092+0000", 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8349999999999998e-06 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524097+0000", 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107565 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530856+0000", 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038358741000000002, 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049176461999999997, 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107565 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530856+0000", 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530856+0000", 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530856+0000", 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8799999999999998e-07 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530859+0000", 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6299999999999998e-06 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530865+0000", 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8490000000000002e-06 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530869+0000", 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6650000000000002e-06 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.209 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107638 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538781+0000", 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030433318000000001, 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041304248000000002, 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.210 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107638 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538781+0000", 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538781+0000", 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538782+0000", 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2399999999999999e-07 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538784+0000", 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4329999999999998e-06 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538791+0000", 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3759999999999996e-06 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538796+0000", 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5849999999999999e-06 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108159 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546688+0000", 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022526485999999998, 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033459788999999997, 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108159 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546688+0000", 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546688+0000", 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546688+0000", 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546690+0000", 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.482e-06 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546693+0000", 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4630000000000001e-06 2024-09-22T01:02:30.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546698+0000", 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6290000000000001e-06 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108190 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551598+0000", 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017616836, 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.02861226, 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108190 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551598+0000", 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551598+0000", 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551598+0000", 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551602+0000", 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3280000000000002e-06 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551636+0000", 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4292999999999998e-05 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551642+0000", 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108341 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560304+0000", 2024-09-22T01:02:30.219 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.008910365, 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019971111999999999, 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108341 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560304+0000", 2024-09-22T01:02:30.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560304+0000", 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560304+0000", 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1400000000000001e-07 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560306+0000", 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3909999999999999e-06 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560309+0000", 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3160000000000001e-06 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560312+0000", 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5390000000000002e-06 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108373 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.574731+0000", 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9944835, 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056063859999999997, 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108373 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.223 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574731+0000", 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574731+0000", 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574731+0000", 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:02:30.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574733+0000", 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4470000000000001e-06 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574736+0000", 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4529999999999998e-06 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574739+0000", 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6769999999999999e-06 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.225 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107457 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523245+0000", 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045969589999999998, 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057142258000000001, 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107457 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523245+0000", 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523245+0000", 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523245+0000", 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2e-07 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523248+0000", 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.548e-06 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523255+0000", 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7859999999999999e-06 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523260+0000", 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5740000000000003e-06 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107508 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524105+0000", 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045109182999999997, 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056344667000000001, 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107508 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524105+0000", 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524105+0000", 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524106+0000", 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2500000000000001e-07 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524108+0000", 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1720000000000001e-06 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524114+0000", 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4829999999999996e-06 2024-09-22T01:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524120+0000", 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3020000000000002e-06 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107567 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530875+0000", 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038339168, 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049633930999999999, 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107567 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530875+0000", 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530875+0000", 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530876+0000", 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-07 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530878+0000", 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9939999999999999e-06 2024-09-22T01:02:30.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530883+0000", 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530888+0000", 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8570000000000003e-06 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107640 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538804+0000", 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030410726999999999, 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041753823000000002, 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107640 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538804+0000", 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538804+0000", 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538804+0000", 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538807+0000", 2024-09-22T01:02:30.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.649e-06 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538814+0000", 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6139999999999997e-06 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538819+0000", 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5790000000000003e-06 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108160 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546702+0000", 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022512151000000001, 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033906693000000002, 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108160 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546702+0000", 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546702+0000", 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546703+0000", 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0699999999999999e-07 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546708+0000", 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4659999999999998e-06 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546712+0000", 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9539999999999998e-06 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546715+0000", 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9500000000000001e-06 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.241 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108191 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.551657+0000", 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017558001, 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028998695000000001, 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108191 2024-09-22T01:02:30.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551657+0000", 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.551657+0000", 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554636+0000", 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029791940000000001 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554642+0000", 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7790000000000001e-06 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554652+0000", 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0118e-05 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554659+0000", 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2960000000000001e-06 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108342 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560315+0000", 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0088993739999999998, 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020404373, 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108342 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560315+0000", 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560315+0000", 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560315+0000", 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0800000000000001e-07 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560317+0000", 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3599999999999999e-06 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560320+0000", 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3450000000000002e-06 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560323+0000", 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.599e-06 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108374 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.574742+0000", 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.994472, 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060387899999999996, 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108374 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574742+0000", 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574742+0000", 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574743+0000", 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2100000000000001e-07 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574744+0000", 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4300000000000001e-06 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574748+0000", 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:02:30.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574750+0000", 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.695e-06 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107459 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523268+0000", 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045946953999999998, 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057578846000000003, 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107459 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523268+0000", 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523268+0000", 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523268+0000", 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5400000000000002e-07 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523270+0000", 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.182e-06 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523277+0000", 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523282+0000", 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6739999999999996e-06 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107509 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524127+0000", 2024-09-22T01:02:30.254 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045087440999999999, 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056780764999999997, 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107509 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524127+0000", 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524127+0000", 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524128+0000", 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524130+0000", 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-22T01:02:30.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524136+0000", 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1720000000000004e-06 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524141+0000", 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107569 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530895+0000", 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038319497000000001, 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050078782000000002, 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107569 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530895+0000", 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530895+0000", 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530895+0000", 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4900000000000001e-07 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530898+0000", 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3379999999999999e-06 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530905+0000", 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7120000000000003e-06 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530910+0000", 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2730000000000002e-06 2024-09-22T01:02:30.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107642 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538826+0000", 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030388288999999999, 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042207455999999997, 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107642 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538826+0000", 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538826+0000", 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538827+0000", 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8200000000000001e-07 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538829+0000", 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3920000000000001e-06 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538837+0000", 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0509999999999997e-06 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538842+0000", 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108161 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546719+0000", 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022495434000000002, 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034380341000000002, 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108161 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546719+0000", 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546719+0000", 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546719+0000", 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546721+0000", 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2920000000000001e-06 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546725+0000", 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0810000000000004e-06 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546728+0000", 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.631e-06 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108192 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554672+0000", 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014542186, 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026489406, 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108192 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554672+0000", 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554672+0000", 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554673+0000", 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554676+0000", 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554683+0000", 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1749999999999999e-06 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554688+0000", 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3430000000000003e-06 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.270 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108343 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560329+0000", 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0088853790000000005, 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020893603, 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108343 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560329+0000", 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560329+0000", 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560329+0000", 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0599999999999999e-07 2024-09-22T01:02:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560331+0000", 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4079999999999999e-06 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560338+0000", 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1250000000000004e-06 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560341+0000", 2024-09-22T01:02:30.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7930000000000002e-06 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108375 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.574754+0000", 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9944611, 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0065397520000000002, 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108375 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574754+0000", 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574754+0000", 2024-09-22T01:02:30.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574754+0000", 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-07 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574759+0000", 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.575e-06 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574763+0000", 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.817e-06 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574766+0000", 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7049999999999999e-06 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107461 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523288+0000", 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045926307, 2024-09-22T01:02:30.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058068186000000001, 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107461 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523288+0000", 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523288+0000", 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523289+0000", 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8799999999999998e-07 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523296+0000", 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1010000000000003e-06 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523303+0000", 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0469999999999996e-06 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523308+0000", 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107510 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524198+0000", 2024-09-22T01:02:30.280 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045016131000000001, 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057220559999999997, 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107510 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524198+0000", 2024-09-22T01:02:30.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524198+0000", 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524199+0000", 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9100000000000004e-07 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524202+0000", 2024-09-22T01:02:30.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.405e-06 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524211+0000", 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6009999999999995e-06 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524218+0000", 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6420000000000001e-06 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107571 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530917+0000", 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038297335000000002, 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050550184999999997, 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107571 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530917+0000", 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530917+0000", 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530918+0000", 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2300000000000002e-07 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530920+0000", 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1280000000000002e-06 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530926+0000", 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.801e-06 2024-09-22T01:02:30.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530930+0000", 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5029999999999999e-06 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107644 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538849+0000", 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030365822000000001, 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042653621000000003, 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107644 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538849+0000", 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538849+0000", 2024-09-22T01:02:30.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538849+0000", 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5900000000000003e-07 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538880+0000", 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0985000000000002e-05 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538888+0000", 2024-09-22T01:02:30.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3429999999999996e-06 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538895+0000", 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.178e-06 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108162 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546732+0000", 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022482434999999999, 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034803343, 2024-09-22T01:02:30.290 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108162 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546732+0000", 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546732+0000", 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546732+0000", 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0800000000000001e-07 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546734+0000", 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546737+0000", 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2679999999999999e-06 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546740+0000", 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2349999999999999e-06 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108193 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554696+0000", 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014518357000000001, 2024-09-22T01:02:30.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026898923000000002, 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108193 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554696+0000", 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554696+0000", 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554697+0000", 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7800000000000002e-07 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554699+0000", 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3640000000000002e-06 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554705+0000", 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4819999999999999e-06 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554711+0000", 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3600000000000004e-06 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108344 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.296 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560345+0000", 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0088696420000000005, 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021310968, 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108344 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560345+0000", 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560345+0000", 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560345+0000", 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3099999999999999e-07 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560347+0000", 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4219999999999999e-06 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560350+0000", 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4259999999999999e-06 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560353+0000", 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8540000000000001e-06 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108376 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.574770+0000", 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9944448, 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069516129999999997, 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108376 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574770+0000", 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574770+0000", 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574770+0000", 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-07 2024-09-22T01:02:30.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574772+0000", 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574775+0000", 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.382e-06 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574778+0000", 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:02:30.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107463 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523316+0000", 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045899044999999999, 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058467581999999997, 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107463 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523316+0000", 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523316+0000", 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523316+0000", 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0600000000000001e-07 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523318+0000", 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0559999999999999e-06 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523324+0000", 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1260000000000003e-06 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523330+0000", 2024-09-22T01:02:30.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6559999999999997e-06 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107511 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524231+0000", 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044983892999999997, 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057620681, 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107511 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524231+0000", 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524231+0000", 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524231+0000", 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3300000000000003e-07 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524235+0000", 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.636e-06 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524248+0000", 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3376e-05 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524257+0000", 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7390000000000005e-06 2024-09-22T01:02:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107573 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530937+0000", 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038277658999999999, 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050974949999999998, 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107573 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530937+0000", 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530937+0000", 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530937+0000", 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:02:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530940+0000", 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3889999999999999e-06 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530946+0000", 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5649999999999998e-06 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530951+0000", 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107651 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538906+0000", 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030309051, 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043071594999999997, 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107651 2024-09-22T01:02:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538906+0000", 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538906+0000", 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538906+0000", 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538909+0000", 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8320000000000002e-06 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538915+0000", 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538921+0000", 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1070000000000004e-06 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108163 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546747+0000", 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022467503, 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035290220999999997, 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108163 2024-09-22T01:02:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546747+0000", 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546747+0000", 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546747+0000", 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:02:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546750+0000", 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4499999999999998e-06 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546757+0000", 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8190000000000003e-06 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546762+0000", 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108194 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554718+0000", 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014496446, 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027385756000000001, 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108194 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554718+0000", 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554718+0000", 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554718+0000", 2024-09-22T01:02:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554721+0000", 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2869999999999999e-06 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554727+0000", 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.596e-06 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554733+0000", 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2079999999999999e-06 2024-09-22T01:02:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108345 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.560356+0000", 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0088582120000000007, 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021807572000000001, 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108345 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560356+0000", 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.560356+0000", 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562721+0000", 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023645839999999999 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562727+0000", 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3450000000000002e-06 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562740+0000", 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3067e-05 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562749+0000", 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6279999999999994e-06 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108377 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.574781+0000", 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9944334, 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.007444165, 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108377 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574781+0000", 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574781+0000", 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574781+0000", 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574783+0000", 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.297e-06 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574786+0000", 2024-09-22T01:02:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3280000000000002e-06 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574791+0000", 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4889999999999997e-06 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107465 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 661504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.523337+0000", 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.045877449000000001, 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058933289, 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107465 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523337+0000", 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523337+0000", 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523338+0000", 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3500000000000002e-07 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523341+0000", 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523348+0000", 2024-09-22T01:02:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8170000000000001e-06 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.523352+0000", 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9189999999999998e-06 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.582180+0000", 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058827982000000001 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.582232+0000", 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1230000000000002e-05 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107512 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524267+0000", 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044947740999999999, 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058042384000000002, 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107512 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524267+0000", 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524267+0000", 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524267+0000", 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524270+0000", 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.717e-06 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524277+0000", 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2389999999999996e-06 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524283+0000", 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2510000000000002e-06 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107575 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530972+0000", 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038242871999999997, 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051368589999999999, 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107575 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530972+0000", 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530972+0000", 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530972+0000", 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.75e-07 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530975+0000", 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7379999999999999e-06 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530983+0000", 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6729999999999992e-06 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530988+0000", 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107654 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538928+0000", 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030286655999999999, 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043442421000000002, 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107654 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538928+0000", 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538928+0000", 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538928+0000", 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538930+0000", 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538936+0000", 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538942+0000", 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8359999999999997e-06 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108164 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546771+0000", 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022443595, 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035657044999999998, 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108164 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546771+0000", 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546771+0000", 2024-09-22T01:02:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546771+0000", 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9299999999999999e-07 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546774+0000", 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.021e-06 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546782+0000", 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5009999999999998e-06 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546787+0000", 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9880000000000004e-06 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108195 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554739+0000", 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014475264, 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027721600999999998, 2024-09-22T01:02:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108195 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554739+0000", 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554739+0000", 2024-09-22T01:02:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554740+0000", 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.34e-07 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554742+0000", 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1289999999999999e-06 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554749+0000", 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6900000000000003e-06 2024-09-22T01:02:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554753+0000", 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9420000000000003e-06 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108346 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.562764+0000", 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0064501489999999996, 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01973024, 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108346 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562764+0000", 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562764+0000", 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562765+0000", 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0800000000000004e-07 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562768+0000", 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2660000000000002e-06 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562777+0000", 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3930000000000008e-06 2024-09-22T01:02:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562783+0000", 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2890000000000003e-06 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108378 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.574794+0000", 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9944205, 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077392320000000004, 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108378 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574794+0000", 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.574794+0000", 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578620+0000", 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003825302 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578626+0000", 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9199999999999998e-06 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578640+0000", 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3762e-05 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578650+0000", 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7489999999999993e-06 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107513 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524290+0000", 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044924620999999998, 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059227687000000001, 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107513 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524290+0000", 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524290+0000", 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524290+0000", 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524293+0000", 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3649999999999998e-06 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524299+0000", 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5459999999999997e-06 2024-09-22T01:02:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524304+0000", 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9520000000000002e-06 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107578 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.530996+0000", 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038218789000000003, 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052586623999999998, 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107578 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530996+0000", 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530996+0000", 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530996+0000", 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.530999+0000", 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3350000000000001e-06 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531004+0000", 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8649999999999998e-06 2024-09-22T01:02:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531009+0000", 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3429999999999998e-06 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107656 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538949+0000", 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030265712, 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044700693, 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107656 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538949+0000", 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538949+0000", 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538949+0000", 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9299999999999999e-07 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538951+0000", 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1560000000000002e-06 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538958+0000", 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.319e-06 2024-09-22T01:02:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538963+0000", 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7600000000000002e-06 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108165 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546793+0000", 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022421330999999999, 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036917088000000001, 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108165 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546793+0000", 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546793+0000", 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546794+0000", 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5199999999999998e-07 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546795+0000", 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.753e-06 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546799+0000", 2024-09-22T01:02:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1300000000000003e-06 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546802+0000", 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.756e-06 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108196 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554763+0000", 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014451258999999999, 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029014410000000001, 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108196 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554763+0000", 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554763+0000", 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554764+0000", 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0999999999999999e-07 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554771+0000", 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.374e-06 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554779+0000", 2024-09-22T01:02:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2789999999999998e-06 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554785+0000", 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.181e-06 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108347 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.562792+0000", 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0064225159999999996, 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021047038000000001, 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108347 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562792+0000", 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562792+0000", 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562793+0000", 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3000000000000001e-07 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562795+0000", 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8260000000000001e-06 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562803+0000", 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1629999999999999e-06 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562808+0000", 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5559999999999998e-06 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108379 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.578666+0000", 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9905486, 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052429570000000003, 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108379 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578666+0000", 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578666+0000", 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578667+0000", 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578670+0000", 2024-09-22T01:02:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.647e-06 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578678+0000", 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2060000000000007e-06 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578684+0000", 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8610000000000003e-06 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107514 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.524311+0000", 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.044903389000000002, 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059661938999999997, 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107514 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524311+0000", 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524311+0000", 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524312+0000", 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524319+0000", 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3590000000000002e-06 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524326+0000", 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1420000000000004e-06 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.524331+0000", 2024-09-22T01:02:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1919999999999996e-06 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107580 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.531017+0000", 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.038197346, 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053020060000000001, 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107580 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531017+0000", 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531017+0000", 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531018+0000", 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531020+0000", 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2570000000000002e-06 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531029+0000", 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6649999999999992e-06 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.531034+0000", 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7060000000000001e-06 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:107660 3.1a 3.3f8d319a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.538969+0000", 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030245412999999999, 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045129006999999999, 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 107660 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538969+0000", 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538969+0000", 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538970+0000", 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1199999999999998e-07 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538972+0000", 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538979+0000", 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1960000000000002e-06 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.538984+0000", 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.853e-06 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108166 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.546807+0000", 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022407336, 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037355396999999999, 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108166 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546807+0000", 2024-09-22T01:02:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546807+0000", 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546808+0000", 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1299999999999999e-07 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546809+0000", 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3510000000000001e-06 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546812+0000", 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4309999999999998e-06 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.546815+0000", 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.757e-06 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108197 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.554793+0000", 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.014422081999999999, 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029427755, 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108197 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554793+0000", 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554793+0000", 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554793+0000", 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3300000000000003e-07 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554796+0000", 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.548e-06 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554802+0000", 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.669e-06 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.554809+0000", 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8360000000000003e-06 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108348 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.562816+0000", 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0063981929999999999, 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021468114999999999, 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108348 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562816+0000", 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562816+0000", 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562817+0000", 2024-09-22T01:02:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562820+0000", 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8159999999999998e-06 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562827+0000", 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3050000000000004e-06 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.562832+0000", 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.446e-06 2024-09-22T01:02:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:108380 3.1a 3.fb85bdda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.578693+0000", 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.990521, 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005660099, 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108380 2024-09-22T01:02:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578693+0000", 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578693+0000", 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578694+0000", 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2500000000000001e-07 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578697+0000", 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9270000000000001e-06 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578704+0000", 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0369999999999997e-06 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.578710+0000", 2024-09-22T01:02:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8209999999999998e-06 2024-09-22T01:02:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 252 2024-09-22T01:02:30.396 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:30.396 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:30.398 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:30.399 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:30.394+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:30.499 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:30.494+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:30.599 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:30.597+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:30.625 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:30.625 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:30.625 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:30.625 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:30.625 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:30.647 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:30.672 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:30.700 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:30.694+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:30.708 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:30.708 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:30.745 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:30.745 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:30.797 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:30.797 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:30.801 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:30.797+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:30.839 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:30.839 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:30.868 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:30.868 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:30.868 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:30.868 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:30.890 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:30.895 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:30.895 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:30.902 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:30.898+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:30.926 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:30.926 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:31.002 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:31.001+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:31.021 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:31.021 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:31.026 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:02:31.028 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-22T01:02:31.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:31.089 INFO:teuthology.orchestra.run.smithi002.stdout:5070 read 0x347e095 thru 0x34823a0 (0x430c bytes) 2024-09-22T01:02:31.090 INFO:teuthology.orchestra.run.smithi002.stdout:5071 read 0x36cd7cf thru 0x36dad22 (0xd554 bytes) 2024-09-22T01:02:31.090 INFO:teuthology.orchestra.run.smithi002.stdout:5072 punch from 0x568cc83 to 0x568e4b4, (0x1831 bytes) 2024-09-22T01:02:31.096 INFO:teuthology.orchestra.run.smithi002.stdout:5073 read 0x5238b21 thru 0x52452d2 (0xc7b2 bytes) 2024-09-22T01:02:31.096 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.096 INFO:teuthology.orchestra.run.smithi002.stdout:5075 writesame 0x1a201a8 thru 0x1a2f679 data_size 0x1(0xf4d2 bytes) 2024-09-22T01:02:31.104 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:31.098+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:31.204 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:31.201+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:31.214 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:31.214 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:31.223 INFO:teuthology.orchestra.run.smithi002.stdout:5076 trunc from 0xa5209dc to 0xd6896b8 2024-09-22T01:02:31.230 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:31.232 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.232 INFO:teuthology.orchestra.run.smithi002.stdout:5080 writesame 0x35ffc1f thru 0x36037fe data_size 0x10(0x3be0 bytes) 2024-09-22T01:02:31.240 INFO:teuthology.orchestra.run.smithi002.stdout:5082 write 0x706bbe0 thru 0x70746ef (0x8b10 bytes) 2024-09-22T01:02:31.263 INFO:teuthology.orchestra.run.smithi002.stdout:5083 read 0x187cc7c thru 0x188c7fe (0xfb83 bytes) 2024-09-22T01:02:31.269 INFO:teuthology.orchestra.run.smithi002.stdout:5084 write 0x5c120e thru 0x5cc84f (0xb642 bytes) 2024-09-22T01:02:31.299 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.299 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.299 INFO:teuthology.orchestra.run.smithi002.stdout:5091 punch from 0x2099979 to 0x20a4679, (0xad00 bytes) 2024-09-22T01:02:31.301 INFO:teuthology.orchestra.run.smithi002.stdout:5092 trunc from 0xd6896b8 to 0xbd18cad 2024-09-22T01:02:31.305 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:31.302+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:31.311 INFO:teuthology.orchestra.run.smithi002.stdout:5093 writesame 0x6409c9a thru 0x640c56c data_size 0x1(0x28d3 bytes) 2024-09-22T01:02:31.328 INFO:teuthology.orchestra.run.smithi002.stdout:5096 read 0x10ea774 thru 0x10f19c4 (0x7251 bytes) 2024-09-22T01:02:31.330 INFO:teuthology.orchestra.run.smithi002.stdout:5097 write 0xaf1cbb thru 0xaf352a (0x1870 bytes) 2024-09-22T01:02:31.365 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:31.365 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:31.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:31.405 INFO:teuthology.orchestra.run.smithi002.stdout:5098 trunc from 0xbd18cad to 0xb67bd25 2024-09-22T01:02:31.405 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:31.401+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:31.414 INFO:teuthology.orchestra.run.smithi002.stdout:5099 read 0x361290 thru 0x362beb (0x195c bytes) 2024-09-22T01:02:31.415 INFO:teuthology.orchestra.run.smithi002.stdout:5100 write 0xb43139b thru 0xb437843 (0x64a9 bytes) 2024-09-22T01:02:31.419 INFO:teuthology.orchestra.run.smithi002.stdout:5101 write 0xcabc3a thru 0xcac3cd (0x794 bytes) 2024-09-22T01:02:31.459 INFO:teuthology.orchestra.run.smithi002.stdout:5102 write 0x148789a thru 0x1492945 (0xb0ac bytes) 2024-09-22T01:02:31.506 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.506 INFO:teuthology.orchestra.run.smithi002.stdout:5105 write 0x264d15b thru 0x26565e5 (0x948b bytes) 2024-09-22T01:02:31.506 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:31.502+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:31.543 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:31.543 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:31.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:31.560 INFO:teuthology.orchestra.run.smithi002.stdout:5106 read 0x6d439e0 thru 0x6d52742 (0xed63 bytes) 2024-09-22T01:02:31.560 INFO:teuthology.orchestra.run.smithi002.stdout:5107 writesame 0x941f727 thru 0x9422ff4 data_size 0xb(0x38ce bytes) 2024-09-22T01:02:31.566 INFO:teuthology.orchestra.run.smithi002.stdout:5111 read 0x2da8ddc thru 0x2db6c94 (0xdeb9 bytes) 2024-09-22T01:02:31.569 INFO:teuthology.orchestra.run.smithi002.stdout:5112 write 0xaaf4484 thru 0xab03da6 (0xf923 bytes) 2024-09-22T01:02:31.585 INFO:teuthology.orchestra.run.smithi002.stdout:5113 trunc from 0xb67bd25 to 0x2786ffb 2024-09-22T01:02:31.645 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:31.642+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:31.693 INFO:teuthology.orchestra.run.smithi002.stdout:5114 writesame 0x41d6b74 thru 0x41e26b3 data_size 0x70(0xbb40 bytes) 2024-09-22T01:02:31.693 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.693 INFO:teuthology.orchestra.run.smithi002.stdout:5116 trunc from 0x41e26b4 to 0x3f59543 2024-09-22T01:02:31.702 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.702 INFO:teuthology.orchestra.run.smithi002.stdout:5118 punch from 0x15f8d15 to 0x1607dcf, (0xf0ba bytes) 2024-09-22T01:02:31.702 INFO:teuthology.orchestra.run.smithi002.stdout:5121 read 0x3938911 thru 0x3947c36 (0xf326 bytes) 2024-09-22T01:02:31.703 INFO:teuthology.orchestra.run.smithi002.stdout:5123 read 0x1f5f47e thru 0x1f6c1d0 (0xcd53 bytes) 2024-09-22T01:02:31.703 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.707 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:31.702+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:31.720 INFO:teuthology.orchestra.run.smithi002.stdout:5125 trunc from 0x3f59543 to 0xcb614ee 2024-09-22T01:02:31.733 INFO:teuthology.orchestra.run.smithi002.stdout:5126 writesame 0x45abf4a thru 0x45b4c79 data_size 0xc(0x8d30 bytes) 2024-09-22T01:02:31.741 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:31.741 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:31.744 INFO:teuthology.orchestra.run.smithi002.stdout:5127 punch from 0xae72491 to 0xae7752e, (0x509d bytes) 2024-09-22T01:02:31.744 INFO:teuthology.orchestra.run.smithi002.stdout:5129 writesame 0x42693db thru 0x4272fbb data_size 0x17(0x9be1 bytes) 2024-09-22T01:02:31.754 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.754 INFO:teuthology.orchestra.run.smithi002.stdout:5132 punch from 0x971e60e to 0x9724ec6, (0x68b8 bytes) 2024-09-22T01:02:31.755 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.755 INFO:teuthology.orchestra.run.smithi002.stdout:5134 read 0xca04cd1 thru 0xca07db8 (0x30e8 bytes) 2024-09-22T01:02:31.755 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.755 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:31.755 INFO:teuthology.orchestra.run.smithi002.stdout:5137 write 0xbafebd2 thru 0xbb0133e (0x276d bytes) 2024-09-22T01:02:31.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:31.763 INFO:teuthology.orchestra.run.smithi002.stdout:5138 write 0x9323985 thru 0x932fd29 (0xc3a5 bytes) 2024-09-22T01:02:31.770 INFO:teuthology.orchestra.run.smithi002.stdout:5139 write 0x1af72ac thru 0x1aff9a6 (0x86fb bytes) 2024-09-22T01:02:31.841 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:31.838+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:31.856 INFO:teuthology.orchestra.run.smithi002.stdout:5141 write 0x481a3ec thru 0x481b714 (0x1329 bytes) 2024-09-22T01:02:31.863 INFO:teuthology.orchestra.run.smithi002.stdout:5142 write 0xafd5a26 thru 0xafd8168 (0x2743 bytes) 2024-09-22T01:02:31.868 INFO:teuthology.orchestra.run.smithi002.stdout:5144 write 0x81c10c0 thru 0x81c93a8 (0x82e9 bytes) 2024-09-22T01:02:31.874 INFO:teuthology.orchestra.run.smithi002.stdout:5145 write 0x17fca47 thru 0x17febcb (0x2185 bytes) 2024-09-22T01:02:31.900 INFO:teuthology.orchestra.run.smithi002.stdout:5146 trunc from 0xcb614ee to 0x3c93fad 2024-09-22T01:02:31.909 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:31.906+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:31.920 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:31.920 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:31.927 INFO:teuthology.orchestra.run.smithi002.stdout:5148 read 0x217228 thru 0x221d7f (0xab58 bytes) 2024-09-22T01:02:31.929 INFO:teuthology.orchestra.run.smithi002.stdout:5149 clone 46 order 21 su 131072 sc 9 2024-09-22T01:02:31.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:32.015 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:02:32.021 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:32.018+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:32.106 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:32.106 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:32.110 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:32.109+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:32.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:32.135 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone45 from 0x3c93fad (overlap 0x2786ffb) to 0x17b7663 2024-09-22T01:02:32.211 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:32.209+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:32.306 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:32.306 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:32.312 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:32.309+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:32.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:32.413 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:32.410+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:32.413 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-22T01:02:32.417 INFO:teuthology.orchestra.run.smithi002.stdout:5150 trunc from 0x3c93fad to 0x38ec9ed 2024-09-22T01:02:32.443 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:32.443 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:32.458 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:32.493 INFO:teuthology.orchestra.run.smithi002.stdout:5153 trunc from 0x38ec9ed to 0xda6d7a4 2024-09-22T01:02:32.505 INFO:teuthology.orchestra.run.smithi002.stdout:5154 write 0x57be908 thru 0x57c5aee (0x71e7 bytes) 2024-09-22T01:02:32.514 INFO:teuthology.orchestra.run.smithi002.stdout:5157 writesame 0x86ce5b1 thru 0x86d7f94 data_size 0x54(0x99e4 bytes) 2024-09-22T01:02:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:5159 write 0x82cfaff thru 0x82dce92 (0xd394 bytes) 2024-09-22T01:02:32.545 INFO:teuthology.orchestra.run.smithi002.stdout:5161 write 0xee17d15 thru 0xee20ce4 (0x8fd0 bytes) 2024-09-22T01:02:32.549 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:32.546+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:32.553 INFO:teuthology.orchestra.run.smithi002.stdout:5162 writesame 0xddd2c87 thru 0xddda21c data_size 0x1d(0x7596 bytes) 2024-09-22T01:02:32.561 INFO:teuthology.orchestra.run.smithi002.stdout:5165 read 0x977eeb6 thru 0x978882c (0x9977 bytes) 2024-09-22T01:02:32.561 INFO:teuthology.orchestra.run.smithi002.stdout:5166 punch from 0x703eed1 to 0x7049477, (0xa5a6 bytes) 2024-09-22T01:02:32.574 INFO:teuthology.orchestra.run.smithi002.stdout:5167 trunc from 0xee20ce5 to 0xa0f84e1 2024-09-22T01:02:32.599 INFO:teuthology.orchestra.run.smithi002.stdout:5168 read 0x2ef6002 thru 0x2ef9399 (0x3398 bytes) 2024-09-22T01:02:32.599 INFO:teuthology.orchestra.run.smithi002.stdout:5169 punch from 0x591c38c to 0x591e4c7, (0x213b bytes) 2024-09-22T01:02:32.599 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:32.599 INFO:teuthology.orchestra.run.smithi002.stdout:5171 punch from 0x2b4e27a to 0x2b51fa0, (0x3d26 bytes) 2024-09-22T01:02:32.599 INFO:teuthology.orchestra.run.smithi002.stdout:5172 read 0x19223a3 thru 0x1928c4a (0x68a8 bytes) 2024-09-22T01:02:32.605 INFO:teuthology.orchestra.run.smithi002.stdout:5173 read 0x1814db thru 0x18ea26 (0xd54c bytes) 2024-09-22T01:02:32.607 INFO:teuthology.orchestra.run.smithi002.stdout:5174 write 0xb35949 thru 0xb45816 (0xfece bytes) 2024-09-22T01:02:32.616 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:32.613+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:32.631 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:32.631 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:32.647 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:32.693 INFO:teuthology.orchestra.run.smithi002.stdout:5177 trunc from 0xa0f84e1 to 0x1872d48 2024-09-22T01:02:32.733 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:32.730+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:32.816 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:32.813+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:32.836 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:32.836 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:32.851 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:32.895 INFO:teuthology.orchestra.run.smithi002.stdout:5178 punch from 0x1401d86 to 0x141138d, (0xf607 bytes) 2024-09-22T01:02:32.917 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:32.913+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:32.920 INFO:teuthology.orchestra.run.smithi002.stdout:5180 write 0xd3f8822 thru 0xd404916 (0xc0f5 bytes) 2024-09-22T01:02:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:5182 read 0x61eb2bc thru 0x61f342d (0x8172 bytes) 2024-09-22T01:02:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:5183 read 0x286db0e thru 0x2874883 (0x6d76 bytes) 2024-09-22T01:02:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:5184 read 0xcddabb2 thru 0xcde2552 (0x79a1 bytes) 2024-09-22T01:02:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:32.932 INFO:teuthology.orchestra.run.smithi002.stdout:5186 trunc from 0xd404917 to 0x82f1471 2024-09-22T01:02:32.947 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:32.950 INFO:teuthology.orchestra.run.smithi002.stdout:5187 trunc from 0x82f1471 to 0xc0e0976 2024-09-22T01:02:32.956 INFO:teuthology.orchestra.run.smithi002.stdout:5188 read 0x67dfa0c thru 0x67e1a63 (0x2058 bytes) 2024-09-22T01:02:32.956 INFO:teuthology.orchestra.run.smithi002.stdout:5189 punch from 0x305b7a0 to 0x30627fa, (0x705a bytes) 2024-09-22T01:02:32.956 INFO:teuthology.orchestra.run.smithi002.stdout:5191 punch from 0x374892f to 0x375185e, (0x8f2f bytes) 2024-09-22T01:02:32.956 INFO:teuthology.orchestra.run.smithi002.stdout:5192 writesame 0x3198af4 thru 0x319e82d data_size 0x1(0x5d3a bytes) 2024-09-22T01:02:32.975 INFO:teuthology.orchestra.run.smithi002.stdout:5193 write 0x2e0dc8f thru 0x2e14232 (0x65a4 bytes) 2024-09-22T01:02:32.985 INFO:teuthology.orchestra.run.smithi002.stdout:5194 trunc from 0xc0e0976 to 0x933beb5 2024-09-22T01:02:32.993 INFO:teuthology.orchestra.run.smithi002.stdout:5195 write 0x48bde98 thru 0x48c5535 (0x769e bytes) 2024-09-22T01:02:32.999 INFO:teuthology.orchestra.run.smithi002.stdout:5196 punch from 0x610a148 to 0x61104ba, (0x6372 bytes) 2024-09-22T01:02:32.999 INFO:teuthology.orchestra.run.smithi002.stdout:5197 read 0x2a78d0d thru 0x2a833e8 (0xa6dc bytes) 2024-09-22T01:02:32.999 INFO:teuthology.orchestra.run.smithi002.stdout:5199 writesame 0x5f69049 thru 0x5f6c9b7 data_size 0xd(0x396f bytes) 2024-09-22T01:02:33.005 INFO:teuthology.orchestra.run.smithi002.stdout:5200 read 0x486ed2c thru 0x4874c83 (0x5f58 bytes) 2024-09-22T01:02:33.005 INFO:teuthology.orchestra.run.smithi002.stdout:5201 write 0x3a978a7 thru 0x3a9b65a (0x3db4 bytes) 2024-09-22T01:02:33.009 INFO:teuthology.orchestra.run.smithi002.stdout:5202 write 0x6cce9b1 thru 0x6cd6a8f (0x80df bytes) 2024-09-22T01:02:33.011 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:33.011 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:33.013 INFO:teuthology.orchestra.run.smithi002.stdout:5204 trunc from 0x933beb5 to 0x932df25 2024-09-22T01:02:33.018 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:33.014+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:33.020 INFO:teuthology.orchestra.run.smithi002.stdout:5206 write 0x23a9968 thru 0x23ae0e4 (0x477d bytes) 2024-09-22T01:02:33.025 INFO:teuthology.orchestra.run.smithi002.stdout:5207 punch from 0x2a08d81 to 0x2a12b3d, (0x9dbc bytes) 2024-09-22T01:02:33.025 INFO:teuthology.orchestra.run.smithi002.stdout:5208 read 0x82f3ef6 thru 0x82fabbc (0x6cc7 bytes) 2024-09-22T01:02:33.025 INFO:teuthology.orchestra.run.smithi002.stdout:5209 punch from 0x66bbd21 to 0x66c2d61, (0x7040 bytes) 2024-09-22T01:02:33.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:33.033 INFO:teuthology.orchestra.run.smithi002.stdout:5210 writesame 0x9764a0f thru 0x976f7df data_size 0x1(0xadd1 bytes) 2024-09-22T01:02:33.046 INFO:teuthology.orchestra.run.smithi002.stdout:5213 write 0xd2fd1e2 thru 0xd303d34 (0x6b53 bytes) 2024-09-22T01:02:33.051 INFO:teuthology.orchestra.run.smithi002.stdout:5214 write 0x9edc360 thru 0x9ee9936 (0xd5d7 bytes) 2024-09-22T01:02:33.056 INFO:teuthology.orchestra.run.smithi002.stdout:5216 write 0x5c8638f thru 0x5c8df45 (0x7bb7 bytes) 2024-09-22T01:02:33.060 INFO:teuthology.orchestra.run.smithi002.stdout:5217 write 0x7a74c23 thru 0x7a81590 (0xc96e bytes) 2024-09-22T01:02:33.068 INFO:teuthology.orchestra.run.smithi002.stdout:5218 trunc from 0xd303d35 to 0xe113cc7 2024-09-22T01:02:33.077 INFO:teuthology.orchestra.run.smithi002.stdout:5220 punch from 0x4f670f to 0x4fc0cb, (0x59bc bytes) 2024-09-22T01:02:33.077 INFO:teuthology.orchestra.run.smithi002.stdout:5221 punch from 0xcc749f5 to 0xcc7753c, (0x2b47 bytes) 2024-09-22T01:02:33.077 INFO:teuthology.orchestra.run.smithi002.stdout:5222 clone 47 order 18 su 131072 sc 15 2024-09-22T01:02:33.114 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:33.114 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:33.119 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:33.117+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:33.131 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:33.131 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:33.147 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:33.147 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:33.164 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:33.165 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:33.185 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:33.185 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:33.193 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:33.193 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:33.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:33.207 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:33.207 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:33.220 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:33.217+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:33.227 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:33.227 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:33.247 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:33.247 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:33.321 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [4, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:02:33.321 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:02:33.321 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-22T01:02:33.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-22T01:02:33.322 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:33.318+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:33.330 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:33.330 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:33.330 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:33.330 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:33.330 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:33.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:33.429 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:33.426+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381858172000001, 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:02:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381802497000002, 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381750152000002, 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381692241000003, 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381633602000001, 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:02:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:02:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.38157837, 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:02:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:02:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381524646000003, 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:02:33.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:02:33.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:02:33.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381452910999997, 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.549 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381395529999999, 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:02:33.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:02:33.558 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381345263999997, 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.559 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381266207000003, 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.564 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:02:33.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381141958000001, 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:02:33.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.381061375999998, 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.574 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:02:33.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.380996187999997, 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:02:33.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:02:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.380221218999999, 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.380171433999998, 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:02:33.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.380090332999998, 2024-09-22T01:02:33.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:02:33.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:02:33.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.599 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.380020348000002, 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:33.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.604 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.379942984000003, 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.609 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.379882463000001, 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:33.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:02:33.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:33.614 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:33.615 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:33.616 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:33.610+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:33.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:33.633 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-22T01:02:33.629+0000 7fa2a81e5640 -1 mon.a@0(leader).osd e107 definitely_dead 0 2024-09-22T01:02:33.717 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:33.713+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:33.758 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:33.850 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:33.849+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:33.919 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:33.914+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:33.930 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:33.931 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:33.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:34.030 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:34.029+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:34.032 INFO:teuthology.orchestra.run.smithi002.stderr:marked in osd.1. 2024-09-22T01:02:34.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:02:34.063 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-22T01:02:34.120 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:34.118+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.621838613000001, 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:02:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:02:34.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.621810770000003, 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:02:34.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:02:34.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.621778554000002, 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.621756816000001, 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.140 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.621737160999999, 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.621706146999998, 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:02:34.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:34.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:02:34.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:02:34.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92263 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 20971520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.666346979, 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049994333000000002, 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.149 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92263 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454537+0000", 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454541+0000", 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7129999999999999e-06 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454551+0000", 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0448000000000001e-05 2024-09-22T01:02:34.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454558+0000", 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.323e-06 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504317+0000", 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049758335000000001 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504408+0000", 2024-09-22T01:02:34.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1508999999999994e-05 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504530+0000", 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122168 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.666301936, 2024-09-22T01:02:34.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:02:34.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.157 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.666262301, 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.158 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.666174860000002, 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:02:34.162 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:34.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:02:34.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:02:34.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.666134640999999, 2024-09-22T01:02:34.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.172 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.666098749, 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.173 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.666051101000001, 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:02:34.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:02:34.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.666016332000002, 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:02:34.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:02:34.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.665983139000002, 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:34.190 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.665938924999999, 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:02:34.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:02:34.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.665896736000001, 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:34.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:34.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:02:34.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.665862623999999, 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.665829696999999, 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:34.207 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.665794929, 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:02:34.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:02:34.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.212 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.212 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:34.212 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:34.221 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:34.218+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:34.321 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:34.318+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:34.330 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:34.330 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:140499 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.273515695, 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27353634999999998, 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for peered", 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140499 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.331 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057286+0000", 2024-09-22T01:02:34.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057297+0000", 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026e-05 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057320+0000", 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3391999999999999e-05 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057331+0000", 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.075e-05 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057403+0000", 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057411+0000", 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2749999999999995e-06 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.335 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:34.335 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 1 2024-09-22T01:02:34.335 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:34.344 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:34.422 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:34.421+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:34.460 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:34.460 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:34.473 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:34.523 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:34.518+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:34.624 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:34.622+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:34.650 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1256 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568501351999998, 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05467234, 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1256 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079336+0000", 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079338+0000", 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.889e-06 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079345+0000", 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:02:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079349+0000", 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5770000000000004e-06 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133858+0000", 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054508501000000001 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.133934+0000", 2024-09-22T01:02:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6607999999999996e-05 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134008+0000", 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4110999999999996e-05 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568482493000005, 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:02:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:02:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568464308000003, 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568446519999995, 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568427962000001, 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:02:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:02:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.56840991, 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:02:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:02:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568392560999996, 2024-09-22T01:02:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568370607999995, 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:02:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568353072999997, 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:02:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568329286999997, 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:02:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:02:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568311425000005, 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:02:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:02:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568284188000007, 2024-09-22T01:02:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:02:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568265483000005, 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:02:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:02:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568241986000004, 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:02:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:02:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568218017000007, 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:02:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.568198801999998, 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:02:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.567194251999993, 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:02:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:02:34.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:02:34.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.564718983999995, 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:02:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:02:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.564688298999997, 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:02:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.849230138999999, 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:02:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:34.739 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:34.740 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:34.741 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:34.738+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:34.841 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:34.837+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:34.871 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:34.871 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:34.885 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:34.970 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:34.969+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:35.045 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:02:35.046 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:35.041+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:35.063 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:35.064 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:35.079 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:35.170 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:35.169+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:35.244 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:35.242+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:35.253 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3.262a7b13 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.480629893, 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366911223, 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771094+0000", 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0930000000000001e-06 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771103+0000", 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0629999999999995e-06 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771127+0000", 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3924e-05 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771147+0000", 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771214+0000", 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7348999999999995e-05 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771220+0000", 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3709999999999996e-06 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.137980+0000", 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366760011 2024-09-22T01:02:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138003+0000", 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2823000000000002e-05 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3.c7ddc9b3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.480542148000001, 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.367176741, 2024-09-22T01:02:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771181+0000", 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771231+0000", 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771247+0000", 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5858999999999998e-05 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771259+0000", 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2814e-05 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771320+0000", 2024-09-22T01:02:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0393999999999998e-05 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771325+0000", 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138340+0000", 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3670150409999999 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138356+0000", 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6782999999999998e-05 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3:c8de5464:::rbd_data.117c2270b223.000000000000001e:head [sparse-read 0~4194304] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.479620449999999, 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366676402, 2024-09-22T01:02:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772103+0000", 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9929999999999998e-06 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772113+0000", 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.499e-06 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772138+0000", 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4743999999999999e-05 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772151+0000", 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3189e-05 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772415+0000", 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026382499999999998 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772427+0000", 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2103000000000001e-05 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138485+0000", 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366057984 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138778+0000", 2024-09-22T01:02:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029306500000000001 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3:cd93bbe3:::rbd_data.117c2270b223.0000000000000038:head [sparse-read 0~3145728] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.479544335, 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366907898, 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5600000000000005e-07 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772183+0000", 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772195+0000", 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1533e-05 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772206+0000", 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1647e-05 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772493+0000", 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028702699999999997 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772511+0000", 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7609999999999999e-05 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138879+0000", 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3663676759999999 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.139085+0000", 2024-09-22T01:02:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020690899999999999 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:90676 3.9 3:963e58c3:::rbd_data.117c77dccaa7.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.709612683, 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093363432999999996, 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 90676 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542112+0000", 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9730000000000002e-06 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542126+0000", 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542161+0000", 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4975000000000003e-05 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542185+0000", 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4613999999999998e-05 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542321+0000", 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013617999999999999 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542562+0000", 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024014899999999999 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.556142+0000", 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013580126 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635270+0000", 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079128575000000007 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635299+0000", 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8996000000000002e-05 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635391+0000", 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1712000000000006e-05 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635473+0000", 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1588999999999994e-05 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104388 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [write 9699485~40067] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.318990638000001, 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896004440000002, 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104388 2024-09-22T01:02:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932733+0000", 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1780000000000002e-06 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932760+0000", 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6174e-05 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932775+0000", 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5603e-05 2024-09-22T01:02:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932787+0000", 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2155e-05 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932873+0000", 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5391999999999998e-05 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932887+0000", 2024-09-22T01:02:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3915e-05 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930412+0000", 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.99752547400000002 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930446+0000", 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3841999999999999e-05 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.019619+0000", 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.089172569 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020035+0000", 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041613899999999999 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020214+0000", 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017929999999999999 2024-09-22T01:02:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.021438+0000", 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012240440000000001 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022156+0000", 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071828699999999996 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022231+0000", 2024-09-22T01:02:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420999999999998e-05 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022332+0000", 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100951 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106194 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 778240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8681480419999996, 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091629434999999995, 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106194 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383577+0000", 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7549999999999999e-06 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383585+0000", 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4209999999999998e-06 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383599+0000", 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4599e-05 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.474958+0000", 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091358710999999995 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475039+0000", 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1075999999999994e-05 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475203+0000", 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016427600000000001 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106195 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8681131989999997, 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091906246999999996, 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106195 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383612+0000", 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383620+0000", 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0600000000000008e-06 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383626+0000", 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8529999999999997e-06 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475245+0000", 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091618623999999996 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475326+0000", 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1166000000000002e-05 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475515+0000", 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018890399999999999 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106196 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8680871759999995, 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092159059000000002, 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106196 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383645+0000", 2024-09-22T01:02:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7119999999999995e-06 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383653+0000", 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.807e-06 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383658+0000", 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.04e-06 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475547+0000", 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091889557999999996 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475630+0000", 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3114000000000004e-05 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475794+0000", 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016325199999999999 2024-09-22T01:02:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106197 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 821248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8680544760000002, 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092422578000000005, 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106197 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383668+0000", 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.63e-07 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383677+0000", 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8859999999999993e-06 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383685+0000", 2024-09-22T01:02:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5609999999999999e-06 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383690+0000", 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475825+0000", 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092134439999999998 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475917+0000", 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2559000000000006e-05 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476090+0000", 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017251899999999999 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106198 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.868021315, 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092626416000000003, 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106198 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384316+0000", 2024-09-22T01:02:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061580100000000002 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384323+0000", 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384338+0000", 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5126999999999999e-05 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384348+0000", 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8479999999999996e-06 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476109+0000", 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091760280999999999 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476178+0000", 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9865999999999998e-05 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476327+0000", 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014848199999999999 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106199 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8673568979999997, 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092274072999999998, 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106199 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385301+0000", 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093565 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385312+0000", 2024-09-22T01:02:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1589e-05 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385342+0000", 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9743999999999998e-05 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385361+0000", 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9219999999999999e-05 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476358+0000", 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090996442999999996 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476438+0000", 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9977999999999999e-05 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476639+0000", 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000201449 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106200 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8663289110000001, 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091589137000000001, 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106200 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385394+0000", 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1599999999999999e-06 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385401+0000", 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9090000000000003e-06 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385420+0000", 2024-09-22T01:02:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385434+0000", 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4068e-05 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476677+0000", 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091242904999999999 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476760+0000", 2024-09-22T01:02:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3783999999999996e-05 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476982+0000", 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022163200000000001 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106201 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8661074260000001, 2024-09-22T01:02:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091715669, 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106201 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385616+0000", 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.277e-06 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385624+0000", 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7910000000000005e-06 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385646+0000", 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2251000000000001e-05 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385680+0000", 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3933000000000002e-05 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477014+0000", 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091334175000000004 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477096+0000", 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2361999999999995e-05 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477330+0000", 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023388 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106202 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8660190090000004, 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091937313000000007, 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106202 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385835+0000", 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013252699999999999 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385844+0000", 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1599999999999998e-06 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385864+0000", 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0579000000000001e-05 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385879+0000", 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4871e-05 2024-09-22T01:02:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477362+0000", 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091483156999999996 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477452+0000", 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9868000000000001e-05 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477640+0000", 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018815100000000001 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106203 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8658001760000005, 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092020950000000004, 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106203 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386095+0000", 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017321200000000001 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386104+0000", 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5109999999999992e-06 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386129+0000", 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4785000000000001e-05 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386145+0000", 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6191000000000001e-05 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477675+0000", 2024-09-22T01:02:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091529570000000005 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477762+0000", 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6861999999999996e-05 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477943+0000", 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180819 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106204 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8655497380000003, 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092064394999999993, 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106204 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386406+0000", 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023343699999999999 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386412+0000", 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386425+0000", 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2719e-05 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386433+0000", 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2810000000000008e-06 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477976+0000", 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091542739999999997 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478069+0000", 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2971000000000001e-05 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478237+0000", 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167949 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106205 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.865274071, 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092018127000000005, 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106205 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387017+0000", 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056957000000000004 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387024+0000", 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387035+0000", 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1187e-05 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387045+0000", 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0750000000000004e-06 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478259+0000", 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091214394000000004 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478311+0000", 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1984000000000002e-05 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478466+0000", 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015504200000000001 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106206 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8646586850000002, 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091679640000000007, 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106206 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387533+0000", 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047020100000000001 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387539+0000", 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8200000000000002e-06 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387552+0000", 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2865e-05 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387560+0000", 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5499999999999997e-06 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478507+0000", 2024-09-22T01:02:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090947741999999998 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478599+0000", 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1181000000000007e-05 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478743+0000", 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014428100000000001 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106208 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8641098820000002, 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091664238999999995, 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106208 2024-09-22T01:02:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391033+0000", 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00342144 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391040+0000", 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391054+0000", 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4089e-05 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391063+0000", 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3809999999999999e-06 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479014+0000", 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087951647999999993 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479099+0000", 2024-09-22T01:02:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4859999999999997e-05 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479276+0000", 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017700699999999999 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:35.350 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:35.351 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:35.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:35.353 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:35.349+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:35.453 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:35.450+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:35.520 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:35.520 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:35.520 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:35.520 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:35.536 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:35.554 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:35.550+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:35.556 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:35.557 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:35.584 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:35.584 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:35.602 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:35.602 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:35.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:35.618 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:35.638 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:35.639 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:35.655 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:35.650+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:35.659 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:35.659 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:35.660 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:35.660 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:35.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:35.683 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:35.684 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:35.706 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:35.706 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:35.774 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:35.773+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:35.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:35.877 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:35.874+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:35.957 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:35.957+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:35.963 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:35.964 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:35.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:36.062 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:36.057+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:36.144 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:36.144 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:36.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:36.160 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:36.154+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:36.227 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone46 from 0xe113cc7 (overlap 0x1872d48) to 0x0 2024-09-22T01:02:36.264 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:36.257+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:36.348 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:36.348 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:36.361 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:36.357+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:36.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:36.461 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:36.458+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:36.466 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-22T01:02:36.498 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:36.498 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:36.521 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:36.569 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:36.566+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:36.632 INFO:teuthology.orchestra.run.smithi002.stdout:5223 read 0xb285737 thru 0xb2907e0 (0xb0aa bytes) 2024-09-22T01:02:36.633 INFO:teuthology.orchestra.run.smithi002.stdout:5224 write 0x3959341 thru 0x3965c0f (0xc8cf bytes) 2024-09-22T01:02:36.642 INFO:teuthology.orchestra.run.smithi002.stdout:5225 read 0x75ddd68 thru 0x75e64ef (0x8788 bytes) 2024-09-22T01:02:36.647 INFO:teuthology.orchestra.run.smithi002.stdout:5227 trunc from 0xe113cc7 to 0xb28d652 2024-09-22T01:02:36.663 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:36.658+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:36.670 INFO:teuthology.orchestra.run.smithi002.stdout:5228 read 0x646c350 thru 0x646e36b (0x201c bytes) 2024-09-22T01:02:36.671 INFO:teuthology.orchestra.run.smithi002.stdout:5229 write 0x9c5b73 thru 0x9c7b3c (0x1fca bytes) 2024-09-22T01:02:36.677 INFO:teuthology.orchestra.run.smithi002.stdout:5230 read 0x27fd566 thru 0x280a16d (0xcc08 bytes) 2024-09-22T01:02:36.677 INFO:teuthology.orchestra.run.smithi002.stdout:5232 read 0x954bac1 thru 0x95507ba (0x4cfa bytes) 2024-09-22T01:02:36.677 INFO:teuthology.orchestra.run.smithi002.stdout:5233 write 0x9446732 thru 0x944bb43 (0x5412 bytes) 2024-09-22T01:02:36.694 INFO:teuthology.orchestra.run.smithi002.stdout:5234 writesame 0xeb78954 thru 0xeb7bfd2 data_size 0x1(0x367f bytes) 2024-09-22T01:02:36.694 INFO:teuthology.orchestra.run.smithi002.stdout:5235 writesame 0x240f834 thru 0x2416a39 data_size 0x2a(0x7206 bytes) 2024-09-22T01:02:36.698 INFO:teuthology.orchestra.run.smithi002.stdout:5236 write 0x636154 thru 0x63886f (0x271c bytes) 2024-09-22T01:02:36.703 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:36.703 INFO:teuthology.orchestra.run.smithi002.stdout:5238 punch from 0x755788 to 0x76010f, (0xa987 bytes) 2024-09-22T01:02:36.703 INFO:teuthology.orchestra.run.smithi002.stdout:5239 punch from 0x148d003 to 0x149a7df, (0xd7dc bytes) 2024-09-22T01:02:36.703 INFO:teuthology.orchestra.run.smithi002.stdout:5241 read 0x33f4210 thru 0x33f6368 (0x2159 bytes) 2024-09-22T01:02:36.703 INFO:teuthology.orchestra.run.smithi002.stdout:5242 punch from 0x836a7be to 0x836cd38, (0x257a bytes) 2024-09-22T01:02:36.704 INFO:teuthology.orchestra.run.smithi002.stdout:5244 read 0x8c61515 thru 0x8c63be0 (0x26cc bytes) 2024-09-22T01:02:36.704 INFO:teuthology.orchestra.run.smithi002.stdout:5247 punch from 0xb11f80b to 0xb12b186, (0xb97b bytes) 2024-09-22T01:02:36.704 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:36.704 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:36.706 INFO:teuthology.orchestra.run.smithi002.stdout:5248 trunc from 0xeb7bfd3 to 0xcf460e4 2024-09-22T01:02:36.714 INFO:teuthology.orchestra.run.smithi002.stdout:5249 writesame 0x65f26de thru 0x6600623 data_size 0x1(0xdf46 bytes) 2024-09-22T01:02:36.720 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:36.765 INFO:teuthology.orchestra.run.smithi002.stdout:5250 write 0x8f60718 thru 0x8f61203 (0xaec bytes) 2024-09-22T01:02:36.773 INFO:teuthology.orchestra.run.smithi002.stdout:5252 punch from 0x6e3131d to 0x6e37081, (0x5d64 bytes) 2024-09-22T01:02:36.782 INFO:teuthology.orchestra.run.smithi002.stdout:5253 trunc from 0xcf460e4 to 0x9a775a0 2024-09-22T01:02:36.805 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:36.802+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:36.837 INFO:teuthology.orchestra.run.smithi002.stdout:5254 trunc from 0x9a775a0 to 0xdc7666a 2024-09-22T01:02:36.847 INFO:teuthology.orchestra.run.smithi002.stdout:5255 write 0xb03143f thru 0xb0399c3 (0x8585 bytes) 2024-09-22T01:02:36.853 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:36.865 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:36.862+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:36.865 INFO:teuthology.orchestra.run.smithi002.stdout:5257 write 0xe4d8c10 thru 0xe4ddc6d (0x505e bytes) 2024-09-22T01:02:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:5258 read 0xc74f5e9 thru 0xc75aed3 (0xb8eb bytes) 2024-09-22T01:02:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:36.872 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:36.872 INFO:teuthology.orchestra.run.smithi002.stdout:5262 write 0x83070c3 thru 0x8310931 (0x986f bytes) 2024-09-22T01:02:36.878 INFO:teuthology.orchestra.run.smithi002.stdout:5264 read 0x9ea37d1 thru 0x9eb0a1c (0xd24c bytes) 2024-09-22T01:02:36.878 INFO:teuthology.orchestra.run.smithi002.stdout:5265 write 0x8cef451 thru 0x8cf3382 (0x3f32 bytes) 2024-09-22T01:02:36.884 INFO:teuthology.orchestra.run.smithi002.stdout:5266 read 0x8b53c04 thru 0x8b5f606 (0xba03 bytes) 2024-09-22T01:02:36.884 INFO:teuthology.orchestra.run.smithi002.stdout:5268 writesame 0x669961c thru 0x66a59bd data_size 0x33(0xc3a2 bytes) 2024-09-22T01:02:36.890 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:36.891 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:36.893 INFO:teuthology.orchestra.run.smithi002.stdout:5270 write 0x9bcc7d8 thru 0x9bd4c85 (0x84ae bytes) 2024-09-22T01:02:36.897 INFO:teuthology.orchestra.run.smithi002.stdout:5271 read 0xbc41379 thru 0xbc4eeec (0xdb74 bytes) 2024-09-22T01:02:36.897 INFO:teuthology.orchestra.run.smithi002.stdout:5272 write 0x7cfe657 thru 0x7d0c7ce (0xe178 bytes) 2024-09-22T01:02:36.904 INFO:teuthology.orchestra.run.smithi002.stdout:5273 read 0xa4e4419 thru 0xa4e66aa (0x2292 bytes) 2024-09-22T01:02:36.904 INFO:teuthology.orchestra.run.smithi002.stdout:5275 read 0xb669dfa thru 0xb672a39 (0x8c40 bytes) 2024-09-22T01:02:36.905 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:36.905 INFO:teuthology.orchestra.run.smithi002.stdout:5277 writesame 0x9e93ad5 thru 0x9e94415 data_size 0x1(0x941 bytes) 2024-09-22T01:02:36.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:36.910 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:36.910 INFO:teuthology.orchestra.run.smithi002.stdout:5282 read 0xbcc485b thru 0xbccd462 (0x8c08 bytes) 2024-09-22T01:02:36.930 INFO:teuthology.orchestra.run.smithi002.stdout:5283 trunc from 0xe4ddc6e to 0x5cec536 2024-09-22T01:02:36.966 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:36.965+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:36.973 INFO:teuthology.orchestra.run.smithi002.stdout:5284 write 0x36b5e28 thru 0x36c18f3 (0xbacc bytes) 2024-09-22T01:02:36.982 INFO:teuthology.orchestra.run.smithi002.stdout:5285 read 0x2d6866f thru 0x2d76607 (0xdf99 bytes) 2024-09-22T01:02:36.982 INFO:teuthology.orchestra.run.smithi002.stdout:5287 write 0x551a00c thru 0x5527f90 (0xdf85 bytes) 2024-09-22T01:02:37.001 INFO:teuthology.orchestra.run.smithi002.stdout:5288 trunc from 0x5cec536 to 0xad1ee4c 2024-09-22T01:02:37.011 INFO:teuthology.orchestra.run.smithi002.stdout:5289 punch from 0xa3b1fd8 to 0xa3bd3c2, (0xb3ea bytes) 2024-09-22T01:02:37.011 INFO:teuthology.orchestra.run.smithi002.stdout:5291 punch from 0x9758afa to 0x9766edb, (0xe3e1 bytes) 2024-09-22T01:02:37.011 INFO:teuthology.orchestra.run.smithi002.stdout:5293 flatten 2024-09-22T01:02:37.067 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:37.062+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:37.070 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:02:37.089 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:37.089 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:37.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:37.193 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:37.190+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:37.268 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:37.262+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:37.274 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:37.274 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:37.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:37.369 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:37.365+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:37.467 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:37.467 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:37.470 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:37.466+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:37.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:37.571 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:37.566+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:37.658 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:37.658 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:37.671 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:37.666+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:37.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:37.726 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:37.733 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:02:37.772 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:37.769+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:37.799 INFO:teuthology.orchestra.run.smithi002.stdout:5295 write 0xd72e455 thru 0xd732505 (0x40b1 bytes) 2024-09-22T01:02:37.802 INFO:teuthology.orchestra.run.smithi002.stdout:5296 writesame 0x8e6bc40 thru 0x8e70910 data_size 0x39(0x4cd1 bytes) 2024-09-22T01:02:37.851 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:37.851 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:37.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:37.873 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:37.869+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:37.899 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:37.900 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:37.922 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:37.941 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:37.959 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:37.974 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:37.970+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:37.982 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:37.982 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:37.987 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:37.987 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:38.002 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:38.007 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:38.007 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:38.030 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:38.053 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:38.053 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:38.075 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:38.070+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:38.141 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:38.142 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:38.142 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:38.142 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:38.159 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:38.202 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:38.201+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:38.277 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:38.274+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:38.290 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:38.291 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:38.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:38.378 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:38.378+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:38.477 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.976945426, 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:02:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:02:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.976917583000002, 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:02:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.976885367000001, 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:02:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.976863629, 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:02:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:02:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.976843973999998, 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.496 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:02:38.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.976812959999997, 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92263 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 20971520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.504 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021453791999999, 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049994333000000002, 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92263 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454537+0000", 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454541+0000", 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7129999999999999e-06 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454551+0000", 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0448000000000001e-05 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454558+0000", 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.323e-06 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504317+0000", 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049758335000000001 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504408+0000", 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1508999999999994e-05 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504530+0000", 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122168 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021408748999999, 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:02:38.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.512 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021369113999999, 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:38.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:02:38.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:02:38.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021281673000001, 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:38.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:02:38.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:02:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.520 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021241453999998, 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:02:38.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:02:38.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.524 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021205561999999, 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:38.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:02:38.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:02:38.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.528 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021157914, 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:38.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021123145000001, 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:02:38.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021089952000001, 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:02:38.537 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:02:38.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.541 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021045738000002, 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.021003549, 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:02:38.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.020969437000002, 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:02:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.020936509999999, 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.020901742, 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:38.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:38.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:38.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:02:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:02:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:38.562 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:38.564 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:38.564 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:38.561+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:38.664 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:38.661+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:38.742 INFO:teuthology.orchestra.run.smithi002.stdout:5297 write 0xb94e8be thru 0xb955243 (0x6986 bytes) 2024-09-22T01:02:38.748 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:38.749 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:38.749 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:38.749 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:38.750 INFO:teuthology.orchestra.run.smithi002.stdout:5298 punch from 0x859479c to 0x8597cbd, (0x3521 bytes) 2024-09-22T01:02:38.750 INFO:teuthology.orchestra.run.smithi002.stdout:5299 punch from 0xcee11f6 to 0xcee42eb, (0x30f5 bytes) 2024-09-22T01:02:38.750 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:38.750 INFO:teuthology.orchestra.run.smithi002.stdout:5301 read 0x266cb5 thru 0x273a45 (0xcd91 bytes) 2024-09-22T01:02:38.751 INFO:teuthology.orchestra.run.smithi002.stdout:5303 read 0x1d74630 thru 0x1d793be (0x4d8f bytes) 2024-09-22T01:02:38.751 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:38.756 INFO:teuthology.orchestra.run.smithi002.stdout:5305 trunc from 0xd732506 to 0xad9a608 2024-09-22T01:02:38.765 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:38.762+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:38.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:38.866 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:38.866+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:38.937 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:38.937 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:38.951 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:38.998 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:38.997+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:39.064 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:02:39.065 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T01:02:39.068 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:39.066+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:39.083 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:39.083 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:39.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:39.083 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001505354, 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:02:39.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:39.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001487169, 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:39.092 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.00146938100001, 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:39.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.096 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001450823, 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:02:39.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:39.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:02:39.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001432771, 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.00141542199999, 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.105 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:39.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.00139346900001, 2024-09-22T01:02:39.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:02:39.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:02:39.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.113 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001375934, 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.117 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001352148, 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.118 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:02:39.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:02:39.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001334286, 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:39.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:02:39.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:02:39.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001307049, 2024-09-22T01:02:39.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:02:39.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:02:39.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001288344, 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:02:39.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:02:39.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:02:39.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001264847, 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.135 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001240878, 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:39.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.001221663, 2024-09-22T01:02:39.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:02:39.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.00021711300001, 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:02:39.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1277 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.99774184499999, 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054804436999999998, 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1277 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083119+0000", 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083128+0000", 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4779999999999997e-06 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083136+0000", 2024-09-22T01:02:39.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9049999999999997e-06 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083141+0000", 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137796+0000", 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054654625999999998 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137865+0000", 2024-09-22T01:02:39.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8991000000000004e-05 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137923+0000", 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.99771115999999, 2024-09-22T01:02:39.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:02:39.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:02:39.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.160 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.282253000000001, 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:02:39.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:02:39.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:140499 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0235768299999997, 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98072477000000002, 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140499 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057286+0000", 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057297+0000", 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026e-05 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057320+0000", 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3391999999999999e-05 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057331+0000", 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.075e-05 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057403+0000", 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057411+0000", 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2749999999999995e-06 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.037974+0000", 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98056295800000004 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.038008+0000", 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3844999999999999e-05 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.171 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:39.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:39.173 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:39.166+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:39.273 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:39.270+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:39.331 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:39.331 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:39.346 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:39.394 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:39.393+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:39.471 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:39.472 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:39.474 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:39.473+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:39.485 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:39.575 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:39.570+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3.262a7b13 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.879499602000003, 2024-09-22T01:02:39.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366911223, 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:39.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771094+0000", 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0930000000000001e-06 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771103+0000", 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0629999999999995e-06 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771127+0000", 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3924e-05 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771147+0000", 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:02:39.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771214+0000", 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7348999999999995e-05 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771220+0000", 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3709999999999996e-06 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.137980+0000", 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366760011 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138003+0000", 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2823000000000002e-05 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3.c7ddc9b3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:39.659 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.879411857000001, 2024-09-22T01:02:39.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.367176741, 2024-09-22T01:02:39.660 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:39.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771181+0000", 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771231+0000", 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771247+0000", 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5858999999999998e-05 2024-09-22T01:02:39.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771259+0000", 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2814e-05 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771320+0000", 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0393999999999998e-05 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771325+0000", 2024-09-22T01:02:39.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138340+0000", 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3670150409999999 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138356+0000", 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6782999999999998e-05 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3:c8de5464:::rbd_data.117c2270b223.000000000000001e:head [sparse-read 0~4194304] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.878490159000002, 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366676402, 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772103+0000", 2024-09-22T01:02:39.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9929999999999998e-06 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772113+0000", 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.499e-06 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772138+0000", 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4743999999999999e-05 2024-09-22T01:02:39.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772151+0000", 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3189e-05 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772415+0000", 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026382499999999998 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:39.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772427+0000", 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2103000000000001e-05 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138485+0000", 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366057984 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138778+0000", 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029306500000000001 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3:cd93bbe3:::rbd_data.117c2270b223.0000000000000038:head [sparse-read 0~3145728] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.878414044000003, 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366907898, 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5600000000000005e-07 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772183+0000", 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772195+0000", 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1533e-05 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772206+0000", 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1647e-05 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772493+0000", 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028702699999999997 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772511+0000", 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7609999999999999e-05 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138879+0000", 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3663676759999999 2024-09-22T01:02:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.139085+0000", 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020690899999999999 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:90676 3.9 3:963e58c3:::rbd_data.117c77dccaa7.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.108482391999999, 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093363432999999996, 2024-09-22T01:02:39.675 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 90676 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542112+0000", 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9730000000000002e-06 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542126+0000", 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:02:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542161+0000", 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4975000000000003e-05 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542185+0000", 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4613999999999998e-05 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542321+0000", 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013617999999999999 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542562+0000", 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024014899999999999 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.556142+0000", 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013580126 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635270+0000", 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079128575000000007 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635299+0000", 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8996000000000002e-05 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635391+0000", 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1712000000000006e-05 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635473+0000", 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1588999999999994e-05 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104388 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [write 9699485~40067] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.717860346999998, 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896004440000002, 2024-09-22T01:02:39.681 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104388 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932733+0000", 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1780000000000002e-06 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932760+0000", 2024-09-22T01:02:39.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6174e-05 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932775+0000", 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5603e-05 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932787+0000", 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2155e-05 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932873+0000", 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5391999999999998e-05 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932887+0000", 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3915e-05 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930412+0000", 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.99752547400000002 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930446+0000", 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3841999999999999e-05 2024-09-22T01:02:39.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.019619+0000", 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.089172569 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020035+0000", 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041613899999999999 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020214+0000", 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017929999999999999 2024-09-22T01:02:39.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.021438+0000", 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012240440000000001 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022156+0000", 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071828699999999996 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022231+0000", 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420999999999998e-05 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022332+0000", 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100951 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106194 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 778240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.267017751000001, 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091629434999999995, 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106194 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383577+0000", 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7549999999999999e-06 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383585+0000", 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4209999999999998e-06 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383599+0000", 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4599e-05 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.474958+0000", 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091358710999999995 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475039+0000", 2024-09-22T01:02:39.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1075999999999994e-05 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475203+0000", 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016427600000000001 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106195 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.266982907999999, 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091906246999999996, 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106195 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383612+0000", 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383620+0000", 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0600000000000008e-06 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383626+0000", 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8529999999999997e-06 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475245+0000", 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091618623999999996 2024-09-22T01:02:39.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475326+0000", 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1166000000000002e-05 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475515+0000", 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018890399999999999 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106196 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.266956885000001, 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092159059000000002, 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106196 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:39.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-22T01:02:39.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383645+0000", 2024-09-22T01:02:39.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7119999999999995e-06 2024-09-22T01:02:39.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383653+0000", 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.807e-06 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383658+0000", 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.04e-06 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475547+0000", 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091889557999999996 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475630+0000", 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3114000000000004e-05 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475794+0000", 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016325199999999999 2024-09-22T01:02:39.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106197 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 821248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.266924185000001, 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092422578000000005, 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106197 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383668+0000", 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.63e-07 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383677+0000", 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8859999999999993e-06 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383685+0000", 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5609999999999999e-06 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383690+0000", 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475825+0000", 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092134439999999998 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475917+0000", 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2559000000000006e-05 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476090+0000", 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017251899999999999 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106198 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.266891024, 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092626416000000003, 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106198 2024-09-22T01:02:39.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384316+0000", 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061580100000000002 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384323+0000", 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384338+0000", 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5126999999999999e-05 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384348+0000", 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8479999999999996e-06 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476109+0000", 2024-09-22T01:02:39.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091760280999999999 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476178+0000", 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9865999999999998e-05 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476327+0000", 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014848199999999999 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106199 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.266226607, 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092274072999999998, 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106199 2024-09-22T01:02:39.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385301+0000", 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093565 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385312+0000", 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1589e-05 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385342+0000", 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9743999999999998e-05 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385361+0000", 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9219999999999999e-05 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476358+0000", 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090996442999999996 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476438+0000", 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9977999999999999e-05 2024-09-22T01:02:39.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476639+0000", 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000201449 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106200 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.26519862, 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091589137000000001, 2024-09-22T01:02:39.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106200 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385394+0000", 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1599999999999999e-06 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385401+0000", 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9090000000000003e-06 2024-09-22T01:02:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385420+0000", 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385434+0000", 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4068e-05 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476677+0000", 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091242904999999999 2024-09-22T01:02:39.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476760+0000", 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3783999999999996e-05 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476982+0000", 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022163200000000001 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106201 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.264977135000001, 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091715669, 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106201 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385616+0000", 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.277e-06 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385624+0000", 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7910000000000005e-06 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385646+0000", 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2251000000000001e-05 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385680+0000", 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3933000000000002e-05 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477014+0000", 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091334175000000004 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477096+0000", 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2361999999999995e-05 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477330+0000", 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023388 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106202 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.264888718, 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091937313000000007, 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106202 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385835+0000", 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013252699999999999 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385844+0000", 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1599999999999998e-06 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385864+0000", 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0579000000000001e-05 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385879+0000", 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4871e-05 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477362+0000", 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091483156999999996 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477452+0000", 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9868000000000001e-05 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477640+0000", 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018815100000000001 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106203 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.264669885, 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092020950000000004, 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106203 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386095+0000", 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017321200000000001 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386104+0000", 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5109999999999992e-06 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386129+0000", 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4785000000000001e-05 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386145+0000", 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6191000000000001e-05 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477675+0000", 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091529570000000005 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477762+0000", 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6861999999999996e-05 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477943+0000", 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180819 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106204 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.735 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.264419447, 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092064394999999993, 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106204 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386406+0000", 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023343699999999999 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386412+0000", 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386425+0000", 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2719e-05 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386433+0000", 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2810000000000008e-06 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477976+0000", 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091542739999999997 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478069+0000", 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2971000000000001e-05 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478237+0000", 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167949 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106205 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.264143779999999, 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092018127000000005, 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106205 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.740 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387017+0000", 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056957000000000004 2024-09-22T01:02:39.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387024+0000", 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387035+0000", 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1187e-05 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387045+0000", 2024-09-22T01:02:39.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0750000000000004e-06 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478259+0000", 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091214394000000004 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478311+0000", 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1984000000000002e-05 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478466+0000", 2024-09-22T01:02:39.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015504200000000001 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106206 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.263528394, 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091679640000000007, 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106206 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387533+0000", 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047020100000000001 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387539+0000", 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8200000000000002e-06 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387552+0000", 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2865e-05 2024-09-22T01:02:39.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387560+0000", 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5499999999999997e-06 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478507+0000", 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090947741999999998 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478599+0000", 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1181000000000007e-05 2024-09-22T01:02:39.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478743+0000", 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014428100000000001 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106208 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.262979591000001, 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091664238999999995, 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106208 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:39.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391033+0000", 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00342144 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391040+0000", 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391054+0000", 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4089e-05 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391063+0000", 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3809999999999999e-06 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479014+0000", 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087951647999999993 2024-09-22T01:02:39.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479099+0000", 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4859999999999997e-05 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479276+0000", 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017700699999999999 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:39.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:39.753 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:39.754 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:39.755 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T01:02:39.755 INFO:teuthology.orchestra.run.smithi002.stdout:5306 write 0x1c4f240 thru 0x1c58f77 (0x9d38 bytes) 2024-09-22T01:02:39.755 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:39.753+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:39.759 INFO:teuthology.orchestra.run.smithi002.stdout:5307 trunc from 0xad9a608 to 0x54fdd75 2024-09-22T01:02:39.766 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T01:02:39.778 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:02:39.801 INFO:teuthology.orchestra.run.smithi002.stdout:5308 trunc from 0x54fdd75 to 0xdd8f478 2024-09-22T01:02:39.821 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:39.821 INFO:teuthology.orchestra.run.smithi002.stdout:5311 writesame 0x66558a1 thru 0x6659313 data_size 0x1(0x3a73 bytes) 2024-09-22T01:02:39.846 INFO:teuthology.orchestra.run.smithi002.stdout:5312 writesame 0x67e4d71 thru 0x67ebf19 data_size 0x3d(0x71a9 bytes) 2024-09-22T01:02:39.851 INFO:teuthology.orchestra.run.smithi002.stdout:5314 punch from 0xc68e7bc to 0xc6942df, (0x5b23 bytes) 2024-09-22T01:02:39.851 INFO:teuthology.orchestra.run.smithi002.stdout:5315 punch from 0x8457eba to 0x845ecce, (0x6e14 bytes) 2024-09-22T01:02:39.851 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:39.851 INFO:teuthology.orchestra.run.smithi002.stdout:5319 write 0x7535bef thru 0x7540dab (0xb1bd bytes) 2024-09-22T01:02:39.855 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:39.850+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:39.857 INFO:teuthology.orchestra.run.smithi002.stdout:5320 writesame 0xbd8c57a thru 0xbd9beb9 data_size 0x20(0xf940 bytes) 2024-09-22T01:02:39.878 INFO:teuthology.orchestra.run.smithi002.stdout:5323 trunc from 0xdd8f478 to 0x3b2d83a 2024-09-22T01:02:39.899 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:39.900 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:39.900 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:39.900 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:39.903 INFO:teuthology.orchestra.run.smithi002.stdout:5324 trunc from 0x3b2d83a to 0x294833f 2024-09-22T01:02:39.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:39.925 INFO:teuthology.orchestra.run.smithi002.stdout:5325 punch from 0x193bfeb to 0x194b5ce, (0xf5e3 bytes) 2024-09-22T01:02:39.925 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:39.946 INFO:teuthology.orchestra.run.smithi002.stdout:5327 write 0x8f14ea1 thru 0x8f21018 (0xc178 bytes) 2024-09-22T01:02:39.954 INFO:teuthology.orchestra.run.smithi002.stdout:5331 write 0x8b501da thru 0x8b58afa (0x8921 bytes) 2024-09-22T01:02:39.955 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:39.949+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:39.960 INFO:teuthology.orchestra.run.smithi002.stdout:5332 write 0x584837b thru 0x5848d35 (0x9bb bytes) 2024-09-22T01:02:39.983 INFO:teuthology.orchestra.run.smithi002.stdout:5333 writesame 0xd3d8d23 thru 0xd3e2b06 data_size 0x2f(0x9de4 bytes) 2024-09-22T01:02:39.983 INFO:teuthology.orchestra.run.smithi002.stdout:5334 write 0x4de131d thru 0x4deda98 (0xc77c bytes) 2024-09-22T01:02:39.991 INFO:teuthology.orchestra.run.smithi002.stdout:5337 read 0x2f29bf5 thru 0x2f3113e (0x754a bytes) 2024-09-22T01:02:39.991 INFO:teuthology.orchestra.run.smithi002.stdout:5339 write 0x27331c2 thru 0x2740fe6 (0xde25 bytes) 2024-09-22T01:02:39.997 INFO:teuthology.orchestra.run.smithi002.stdout:5340 writesame 0x6816e48 thru 0x681f378 data_size 0x1(0x8531 bytes) 2024-09-22T01:02:40.046 INFO:teuthology.orchestra.run.smithi002.stdout:5341 writesame 0xbb2a078 thru 0xbb393fe data_size 0x9(0xf387 bytes) 2024-09-22T01:02:40.056 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:40.049+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:40.063 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:40.064 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:40.068 INFO:teuthology.orchestra.run.smithi002.stdout:5342 write 0xe614f33 thru 0xe61e03e (0x910c bytes) 2024-09-22T01:02:40.075 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:40.075 INFO:teuthology.orchestra.run.smithi002.stdout:5343 write 0xcd91aa thru 0xcdea72 (0x58c9 bytes) 2024-09-22T01:02:40.080 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:40.080 INFO:teuthology.orchestra.run.smithi002.stdout:5344 read 0xbcc4112 thru 0xbccc406 (0x82f5 bytes) 2024-09-22T01:02:40.080 INFO:teuthology.orchestra.run.smithi002.stdout:5345 read 0x9dc5fb5 thru 0x9dd141c (0xb468 bytes) 2024-09-22T01:02:40.094 INFO:teuthology.orchestra.run.smithi002.stdout:5347 trunc from 0xe61e03f to 0x581e723 2024-09-22T01:02:40.118 INFO:teuthology.orchestra.run.smithi002.stdout:5349 writesame 0xed5b7c thru 0xed6f9b data_size 0x2e(0x1420 bytes) 2024-09-22T01:02:40.126 INFO:teuthology.orchestra.run.smithi002.stdout:5351 write 0x2d0f8c2 thru 0x2d1a82a (0xaf69 bytes) 2024-09-22T01:02:40.131 INFO:teuthology.orchestra.run.smithi002.stdout:5352 writesame 0x405287c thru 0x4059a70 data_size 0x1(0x71f5 bytes) 2024-09-22T01:02:40.157 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:40.153+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:40.157 INFO:teuthology.orchestra.run.smithi002.stdout:5353 write 0x2c1d654 thru 0x2c27578 (0x9f25 bytes) 2024-09-22T01:02:40.166 INFO:teuthology.orchestra.run.smithi002.stdout:5354 read 0x50dabc0 thru 0x50dac11 (0x52 bytes) 2024-09-22T01:02:40.166 INFO:teuthology.orchestra.run.smithi002.stdout:5355 read 0x470d2fa thru 0x4719aeb (0xc7f2 bytes) 2024-09-22T01:02:40.191 INFO:teuthology.orchestra.run.smithi002.stdout:5356 writesame 0xeccd4f2 thru 0xecd2f1c data_size 0x1(0x5a2b bytes) 2024-09-22T01:02:40.202 INFO:teuthology.orchestra.run.smithi002.stdout:5358 write 0xed680b8 thru 0xed730af (0xaff8 bytes) 2024-09-22T01:02:40.207 INFO:teuthology.orchestra.run.smithi002.stdout:5361 write 0x695b0a2 thru 0x6966650 (0xb5af bytes) 2024-09-22T01:02:40.214 INFO:teuthology.orchestra.run.smithi002.stdout:5362 write 0xac8d39e thru 0xac8db88 (0x7eb bytes) 2024-09-22T01:02:40.221 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.221 INFO:teuthology.orchestra.run.smithi002.stdout:5364 read 0x523fa8d thru 0x524cfc0 (0xd534 bytes) 2024-09-22T01:02:40.221 INFO:teuthology.orchestra.run.smithi002.stdout:5365 punch from 0x2415bc7 to 0x241add6, (0x520f bytes) 2024-09-22T01:02:40.221 INFO:teuthology.orchestra.run.smithi002.stdout:5366 read 0x8687a1b thru 0x8692012 (0xa5f8 bytes) 2024-09-22T01:02:40.221 INFO:teuthology.orchestra.run.smithi002.stdout:5367 write 0x3e7b0c8 thru 0x3e880b3 (0xcfec bytes) 2024-09-22T01:02:40.227 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:40.228 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:40.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:40.228 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:40.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.228 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.228 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:40.228 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752678953, 2024-09-22T01:02:40.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:40.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:40.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752658124, 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:40.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752629609000003, 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.238 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:40.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752606958999998, 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:40.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:40.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752584964, 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:40.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:40.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752559900000001, 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:40.251 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:40.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752537467000003, 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:40.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:40.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752515989999999, 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:40.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.752487232, 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:40.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.75167424, 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.751649555, 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:40.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747328021000001, 2024-09-22T01:02:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:40.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:40.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747299169999998, 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.288 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747277445000002, 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:40.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:40.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747257015000002, 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.293 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:40.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:40.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747234699000003, 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:40.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.302 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747166903999997, 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:40.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747133767000001, 2024-09-22T01:02:40.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:40.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:40.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747109844999997, 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.312 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:40.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.313 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:40.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:40.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.747086248000002, 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:40.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:40.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:40.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:40.322 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:40.322 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.322 INFO:teuthology.orchestra.run.smithi002.stdout:5370 punch from 0x31e5f29 to 0x31eee8e, (0x8f65 bytes) 2024-09-22T01:02:40.322 INFO:teuthology.orchestra.run.smithi002.stdout:5372 write 0xbbff645 thru 0xbc01d8e (0x274a bytes) 2024-09-22T01:02:40.323 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:40.323 INFO:teuthology.orchestra.run.smithi002.stdout:5373 write 0x7f1e915 thru 0x7f25948 (0x7034 bytes) 2024-09-22T01:02:40.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:40.325 INFO:teuthology.orchestra.run.smithi002.stdout:5374 trunc from 0xed730b0 to 0xa001e25 2024-09-22T01:02:40.325 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:40.325 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:40.325 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:40.326 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:40.326 INFO:teuthology.orchestra.run.smithi002.stdout:5376 trunc from 0xa001e25 to 0x526d2a5 2024-09-22T01:02:40.327 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:40.327 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:40.328 INFO:teuthology.orchestra.run.smithi002.stdout:5378 punch from 0x171f714 to 0x172dcec, (0xe5d8 bytes) 2024-09-22T01:02:40.328 INFO:teuthology.orchestra.run.smithi002.stdout:5379 read 0x4d047e0 thru 0x4d0b239 (0x6a5a bytes) 2024-09-22T01:02:40.328 INFO:teuthology.orchestra.run.smithi002.stdout:5380 read 0x513ca48 thru 0x5143757 (0x6d10 bytes) 2024-09-22T01:02:40.328 INFO:teuthology.orchestra.run.smithi002.stdout:5382 punch from 0x1468ae9 to 0x147166e, (0x8b85 bytes) 2024-09-22T01:02:40.328 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.328 INFO:teuthology.orchestra.run.smithi002.stdout:5386 read 0x375ac6c thru 0x3768379 (0xd70e bytes) 2024-09-22T01:02:40.328 INFO:teuthology.orchestra.run.smithi002.stdout:5387 read 0x1c47f7a thru 0x1c54a5e (0xcae5 bytes) 2024-09-22T01:02:40.328 INFO:teuthology.orchestra.run.smithi002.stdout:5388 trunc from 0x526d2a5 to 0xdf41ff 2024-09-22T01:02:40.329 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:40.334 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:40.334 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:40.333+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:40.348 INFO:teuthology.orchestra.run.smithi002.stdout:5389 trunc from 0xdf41ff to 0x918fcdd 2024-09-22T01:02:40.355 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:40.355 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:40.359 INFO:teuthology.orchestra.run.smithi002.stdout:5391 trunc from 0x918fcdd to 0xb056cfa 2024-09-22T01:02:40.367 INFO:teuthology.orchestra.run.smithi002.stdout:5392 read 0x8ee8385 thru 0x8eeaccf (0x294b bytes) 2024-09-22T01:02:40.367 INFO:teuthology.orchestra.run.smithi002.stdout:5393 read 0x3fdc130 thru 0x3fdeeb9 (0x2d8a bytes) 2024-09-22T01:02:40.375 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:40.375 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:40.381 INFO:teuthology.orchestra.run.smithi002.stdout:5394 write 0xeda5fc5 thru 0xedb52d8 (0xf314 bytes) 2024-09-22T01:02:40.385 INFO:teuthology.orchestra.run.smithi002.stdout:5395 writesame 0x4de41a1 thru 0x4df0607 data_size 0x1(0xc467 bytes) 2024-09-22T01:02:40.416 INFO:teuthology.orchestra.run.smithi002.stdout:5396 write 0xd17724f thru 0xd182bcb (0xb97d bytes) 2024-09-22T01:02:40.428 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:40.421+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:40.429 INFO:teuthology.orchestra.run.smithi002.stdout:5397 trunc from 0xedb52d9 to 0xa4fe194 2024-09-22T01:02:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:5398 read 0x7bd9064 thru 0x7bdc314 (0x32b1 bytes) 2024-09-22T01:02:40.463 INFO:teuthology.orchestra.run.smithi002.stdout:5400 writesame 0xc65133f thru 0xc652ea6 data_size 0x4(0x1b68 bytes) 2024-09-22T01:02:40.463 INFO:teuthology.orchestra.run.smithi002.stdout:5401 write 0xb32f95f thru 0xb3355f1 (0x5c93 bytes) 2024-09-22T01:02:40.465 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:40.465 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:40.468 INFO:teuthology.orchestra.run.smithi002.stdout:5402 write 0x9063370 thru 0x9069f6d (0x6bfe bytes) 2024-09-22T01:02:40.473 INFO:teuthology.orchestra.run.smithi002.stdout:5403 writesame 0xce1a3d thru 0xce392d data_size 0x1(0x1ef1 bytes) 2024-09-22T01:02:40.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:40.482 INFO:teuthology.orchestra.run.smithi002.stdout:5405 punch from 0x178bef1 to 0x179846d, (0xc57c bytes) 2024-09-22T01:02:40.483 INFO:teuthology.orchestra.run.smithi002.stdout:5406 writesame 0xa549990 thru 0xa55345e data_size 0x1(0x9acf bytes) 2024-09-22T01:02:40.512 INFO:teuthology.orchestra.run.smithi002.stdout:5407 read 0x9a49206 thru 0x9a4a451 (0x124c bytes) 2024-09-22T01:02:40.512 INFO:teuthology.orchestra.run.smithi002.stdout:5409 write 0x1845c22 thru 0x184be47 (0x6226 bytes) 2024-09-22T01:02:40.519 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.529 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:40.525+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:40.532 INFO:teuthology.orchestra.run.smithi002.stdout:5411 write 0xd9ef456 thru 0xd9efdaf (0x95a bytes) 2024-09-22T01:02:40.538 INFO:teuthology.orchestra.run.smithi002.stdout:5412 read 0x1950c4f thru 0x1952b22 (0x1ed4 bytes) 2024-09-22T01:02:40.539 INFO:teuthology.orchestra.run.smithi002.stdout:5413 write 0xc341e42 thru 0xc3438af (0x1a6e bytes) 2024-09-22T01:02:40.544 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.545 INFO:teuthology.orchestra.run.smithi002.stdout:5417 read 0x58c8cff thru 0x58d6167 (0xd469 bytes) 2024-09-22T01:02:40.545 INFO:teuthology.orchestra.run.smithi002.stdout:5418 writesame 0x11c668 thru 0x127a49 data_size 0x4b(0xb3e2 bytes) 2024-09-22T01:02:40.549 INFO:teuthology.orchestra.run.smithi002.stdout:5419 writesame 0x19bc7fd thru 0x19bd838 data_size 0x2(0x103c bytes) 2024-09-22T01:02:40.556 INFO:teuthology.orchestra.run.smithi002.stdout:5420 write 0x8df63d5 thru 0x8dfb7eb (0x5417 bytes) 2024-09-22T01:02:40.589 INFO:teuthology.orchestra.run.smithi002.stdout:5421 trunc from 0xd9efdb0 to 0x2c285cc 2024-09-22T01:02:40.612 INFO:teuthology.orchestra.run.smithi002.stdout:5423 read 0x240ebde thru 0x2411f61 (0x3384 bytes) 2024-09-22T01:02:40.612 INFO:teuthology.orchestra.run.smithi002.stdout:5424 punch from 0x8b23aa to 0x8bf35d, (0xcfb3 bytes) 2024-09-22T01:02:40.624 INFO:teuthology.orchestra.run.smithi002.stdout:5425 write 0x2fbc6e5 thru 0x2fcb9cf (0xf2eb bytes) 2024-09-22T01:02:40.630 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:40.629+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:40.633 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.633 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.633 INFO:teuthology.orchestra.run.smithi002.stdout:5428 read 0x6239f thru 0x644bd (0x211f bytes) 2024-09-22T01:02:40.633 INFO:teuthology.orchestra.run.smithi002.stdout:5429 read 0x26c5012 thru 0x26c9d81 (0x4d70 bytes) 2024-09-22T01:02:40.657 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:40.657 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:40.671 INFO:teuthology.orchestra.run.smithi002.stdout:5431 write 0xb98518b thru 0xb98ae01 (0x5c77 bytes) 2024-09-22T01:02:40.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:40.677 INFO:teuthology.orchestra.run.smithi002.stdout:5432 read 0x10f9f55 thru 0x10fbfdd (0x2089 bytes) 2024-09-22T01:02:40.693 INFO:teuthology.orchestra.run.smithi002.stdout:5435 trunc from 0xb98ae02 to 0x6294e15 2024-09-22T01:02:40.714 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.730 INFO:teuthology.orchestra.run.smithi002.stdout:5438 trunc from 0x6294e15 to 0xc15a2fc 2024-09-22T01:02:40.731 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:40.725+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T01:02:40.747 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:02:40.757 INFO:teuthology.orchestra.run.smithi002.stdout:5439 trunc from 0xc15a2fc to 0x5b29e71 2024-09-22T01:02:40.770 INFO:teuthology.orchestra.run.smithi002.stdout:5440 read 0x4b3eec2 thru 0x4b43933 (0x4a72 bytes) 2024-09-22T01:02:40.770 INFO:teuthology.orchestra.run.smithi002.stdout:5441 read 0x58ead81 thru 0x58eb267 (0x4e7 bytes) 2024-09-22T01:02:40.770 INFO:teuthology.orchestra.run.smithi002.stdout:5442 punch from 0x5acc4d4 to 0x5ad8674, (0xc1a0 bytes) 2024-09-22T01:02:40.777 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T01:02:40.790 INFO:teuthology.orchestra.run.smithi002.stdout:5443 write 0xc36758e thru 0xc372083 (0xaaf6 bytes) 2024-09-22T01:02:40.797 INFO:teuthology.orchestra.run.smithi002.stdout:5444 write 0x7a502af thru 0x7a5acb2 (0xaa04 bytes) 2024-09-22T01:02:40.801 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:40.801 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:40.802 INFO:teuthology.orchestra.run.smithi002.stdout:5445 punch from 0x5472cd6 to 0x547a827, (0x7b51 bytes) 2024-09-22T01:02:40.802 INFO:teuthology.orchestra.run.smithi002.stdout:5446 punch from 0x3e37258 to 0x3e41941, (0xa6e9 bytes) 2024-09-22T01:02:40.802 INFO:teuthology.orchestra.run.smithi002.stdout:5449 write 0xb762f24 thru 0xb77227f (0xf35c bytes) 2024-09-22T01:02:40.807 INFO:teuthology.orchestra.run.smithi002.stdout:5450 write 0x6eb854d thru 0x6ec31be (0xac72 bytes) 2024-09-22T01:02:40.813 INFO:teuthology.orchestra.run.smithi002.stdout:5452 read 0x209ae96 thru 0x20a8d69 (0xded4 bytes) 2024-09-22T01:02:40.813 INFO:teuthology.orchestra.run.smithi002.stdout:5453 trunc from 0xc372084 to 0xc05e65a 2024-09-22T01:02:40.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:40.829 INFO:teuthology.orchestra.run.smithi002.stdout:5454 read 0xa9e5d5a thru 0xa9e8456 (0x26fd bytes) 2024-09-22T01:02:40.829 INFO:teuthology.orchestra.run.smithi002.stdout:5455 writesame 0x550c875 thru 0x551246b data_size 0x1(0x5bf7 bytes) 2024-09-22T01:02:40.831 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:40.829+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:40.867 INFO:teuthology.orchestra.run.smithi002.stdout:5456 trunc from 0xc05e65a to 0x4e9d3d 2024-09-22T01:02:40.897 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.897 INFO:teuthology.orchestra.run.smithi002.stdout:5460 punch from 0x4e06a7 to 0x4e9d3d, (0x9696 bytes) 2024-09-22T01:02:40.897 INFO:teuthology.orchestra.run.smithi002.stdout:5461 read 0xab6a5 thru 0xb1fd0 (0x692c bytes) 2024-09-22T01:02:40.912 INFO:teuthology.orchestra.run.smithi002.stdout:5462 trunc from 0x4e9d3d to 0x90f71d0 2024-09-22T01:02:40.921 INFO:teuthology.orchestra.run.smithi002.stdout:5463 read 0x46b7122 thru 0x46c53ac (0xe28b bytes) 2024-09-22T01:02:40.921 INFO:teuthology.orchestra.run.smithi002.stdout:5464 read 0x4ded4fb thru 0x4dfc018 (0xeb1e bytes) 2024-09-22T01:02:40.921 INFO:teuthology.orchestra.run.smithi002.stdout:5470 writesame 0x513ea32 thru 0x5149f89 data_size 0x1c(0xb558 bytes) 2024-09-22T01:02:40.926 INFO:teuthology.orchestra.run.smithi002.stdout:5471 read 0xa92ca6 thru 0xa94a4a (0x1da5 bytes) 2024-09-22T01:02:40.932 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:40.929+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:40.938 INFO:teuthology.orchestra.run.smithi002.stdout:5472 writesame 0xbb7d3b4 thru 0xbb8858c data_size 0x1(0xb1d9 bytes) 2024-09-22T01:02:40.938 INFO:teuthology.orchestra.run.smithi002.stdout:5475 read 0x7fddc31 thru 0x7fe5d94 (0x8164 bytes) 2024-09-22T01:02:40.938 INFO:teuthology.orchestra.run.smithi002.stdout:5476 read 0x5bdf21b thru 0x5be3ec7 (0x4cad bytes) 2024-09-22T01:02:40.938 INFO:teuthology.orchestra.run.smithi002.stdout:5477 writesame 0x15b28ac thru 0x15bb04b data_size 0x46(0x87a0 bytes) 2024-09-22T01:02:40.950 INFO:teuthology.orchestra.run.smithi002.stdout:5478 write 0xcd350a5 thru 0xcd3b3e8 (0x6344 bytes) 2024-09-22T01:02:40.954 INFO:teuthology.orchestra.run.smithi002.stdout:5479 write 0xb68eeba thru 0xb69e5aa (0xf6f1 bytes) 2024-09-22T01:02:40.959 INFO:teuthology.orchestra.run.smithi002.stdout:5481 read 0x4913768 thru 0x49169fb (0x3294 bytes) 2024-09-22T01:02:40.959 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.959 INFO:teuthology.orchestra.run.smithi002.stdout:5484 write 0x6fba63a thru 0x6fbd8fd (0x32c4 bytes) 2024-09-22T01:02:40.963 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:40.963 INFO:teuthology.orchestra.run.smithi002.stdout:5487 write 0x1607b6d thru 0x1608bf3 (0x1087 bytes) 2024-09-22T01:02:40.967 INFO:teuthology.orchestra.run.smithi002.stdout:5488 read 0xa6dbdf thru 0xa6eec2 (0x12e4 bytes) 2024-09-22T01:02:40.968 INFO:teuthology.orchestra.run.smithi002.stdout:5489 write 0x38c6466 thru 0x38d496e (0xe509 bytes) 2024-09-22T01:02:40.972 INFO:teuthology.orchestra.run.smithi002.stdout:5490 punch from 0x6f80d4a to 0x6f8520d, (0x44c3 bytes) 2024-09-22T01:02:40.972 INFO:teuthology.orchestra.run.smithi002.stdout:5491 punch from 0x575f9d2 to 0x5767e53, (0x8481 bytes) 2024-09-22T01:02:40.972 INFO:teuthology.orchestra.run.smithi002.stdout:5492 write 0x3f92dde thru 0x3f9f438 (0xc65b bytes) 2024-09-22T01:02:40.976 INFO:teuthology.orchestra.run.smithi002.stdout:5493 read 0x7778392 thru 0x777baeb (0x375a bytes) 2024-09-22T01:02:40.977 INFO:teuthology.orchestra.run.smithi002.stdout:5494 trunc from 0xcd3b3e9 to 0xc4e183f 2024-09-22T01:02:40.983 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:40.983 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:40.991 INFO:teuthology.orchestra.run.smithi002.stdout:5495 punch from 0x513c1f2 to 0x5143182, (0x6f90 bytes) 2024-09-22T01:02:40.991 INFO:teuthology.orchestra.run.smithi002.stdout:5496 write 0x13a4621 thru 0x13a4ef1 (0x8d1 bytes) 2024-09-22T01:02:40.995 INFO:teuthology.orchestra.run.smithi002.stdout:5498 punch from 0x3aad42a to 0x3ab4c5c, (0x7832 bytes) 2024-09-22T01:02:40.995 INFO:teuthology.orchestra.run.smithi002.stdout:5499 write 0x310c2c1 thru 0x3110677 (0x43b7 bytes) 2024-09-22T01:02:40.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:41.011 INFO:teuthology.orchestra.run.smithi002.stdout:5500 trunc from 0xc4e183f to 0x4ca15c3 2024-09-22T01:02:41.027 INFO:teuthology.orchestra.run.smithi002.stdout:5501 writesame 0x346995c thru 0x346d66d data_size 0x1(0x3d12 bytes) 2024-09-22T01:02:41.038 INFO:teuthology.orchestra.run.smithi002.stdout:5502 punch from 0x26b66cd to 0x26c501a, (0xe94d bytes) 2024-09-22T01:02:41.038 INFO:teuthology.orchestra.run.smithi002.stdout:5503 punch from 0x3cfad05 to 0x3d07337, (0xc632 bytes) 2024-09-22T01:02:41.038 INFO:teuthology.orchestra.run.smithi002.stdout:5504 write 0x1bfe42 thru 0x1ca940 (0xaaff bytes) 2024-09-22T01:02:41.041 INFO:teuthology.orchestra.run.smithi002.stdout:5506 write 0x198d4de thru 0x199c603 (0xf126 bytes) 2024-09-22T01:02:41.041 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:41.037+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:41.045 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:41.045 INFO:teuthology.orchestra.run.smithi002.stdout:5508 writesame 0x198e8c6 thru 0x1998cd1 data_size 0x2(0xa40c bytes) 2024-09-22T01:02:41.061 INFO:teuthology.orchestra.run.smithi002.stdout:5509 writesame 0x41da190 thru 0x41e0883 data_size 0x16(0x66f4 bytes) 2024-09-22T01:02:41.089 INFO:teuthology.orchestra.run.smithi002.stdout:5511 writesame 0x95d1761 thru 0x95d8214 data_size 0x2(0x6ab4 bytes) 2024-09-22T01:02:41.103 INFO:teuthology.orchestra.run.smithi002.stdout:5512 write 0xa8ea0b5 thru 0xa8f3daa (0x9cf6 bytes) 2024-09-22T01:02:41.117 INFO:teuthology.orchestra.run.smithi002.stdout:5513 trunc from 0xa8f3dab to 0xe137617 2024-09-22T01:02:41.127 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:41.127 INFO:teuthology.orchestra.run.smithi002.stdout:5516 punch from 0x8153ab6 to 0x8155eb4, (0x23fe bytes) 2024-09-22T01:02:41.127 INFO:teuthology.orchestra.run.smithi002.stdout:5517 write 0x1d12e35 thru 0x1d194fe (0x66ca bytes) 2024-09-22T01:02:41.133 INFO:teuthology.orchestra.run.smithi002.stdout:5518 write 0x48630c thru 0x48779c (0x1491 bytes) 2024-09-22T01:02:41.135 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:41.133+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:41.139 INFO:teuthology.orchestra.run.smithi002.stdout:5519 write 0x5c1e923 thru 0x5c21737 (0x2e15 bytes) 2024-09-22T01:02:41.152 INFO:teuthology.orchestra.run.smithi002.stdout:5520 trunc from 0xe137617 to 0x8e92678 2024-09-22T01:02:41.171 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:41.171 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:41.171 INFO:teuthology.orchestra.run.smithi002.stdout:5521 read 0x53fed87 thru 0x5404d06 (0x5f80 bytes) 2024-09-22T01:02:41.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:41.192 INFO:teuthology.orchestra.run.smithi002.stdout:5523 write 0xd2f569b thru 0xd302e9d (0xd803 bytes) 2024-09-22T01:02:41.199 INFO:teuthology.orchestra.run.smithi002.stdout:5524 punch from 0xb31b9b3 to 0xb31d0f2, (0x173f bytes) 2024-09-22T01:02:41.199 INFO:teuthology.orchestra.run.smithi002.stdout:5525 clone 48 order 19 su 131072 sc 2 2024-09-22T01:02:41.236 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:41.229+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:41.310 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:41.311 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:41.325 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:41.369 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:41.365+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:41.437 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:41.437+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:41.495 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:41.495 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:41.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:41.538 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:41.533+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:41.627 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:41.627 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:41.638 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:41.633+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:41.641 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:41.740 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:41.733+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:41.764 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T01:02:41.795 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-22T01:02:41.821 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:41.821 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:41.836 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:41.882 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:41.881+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:41.942 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:41.937+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:42.001 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:42.001 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:42.016 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:42.042 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:42.041+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:42.055 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-22T01:02:42.058 INFO:teuthology.orchestra.run.smithi002.stdout:5526 punch from 0x587e827 to 0x5887a6c, (0x9245 bytes) 2024-09-22T01:02:42.071 INFO:teuthology.orchestra.run.smithi002.stdout:5527 write 0xd432fb5 thru 0xd437e01 (0x4e4d bytes) 2024-09-22T01:02:42.076 INFO:teuthology.orchestra.run.smithi002.stdout:5529 writesame 0xa9579a7 thru 0xa96757f data_size 0x1(0xfbd9 bytes) 2024-09-22T01:02:42.124 INFO:teuthology.orchestra.run.smithi002.stdout:5532 punch from 0x4f33104 to 0x4f33e87, (0xd83 bytes) 2024-09-22T01:02:42.125 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.125 INFO:teuthology.orchestra.run.smithi002.stdout:5535 writesame 0xb4b7834 thru 0xb4c45f7 data_size 0x1a(0xcdc4 bytes) 2024-09-22T01:02:42.136 INFO:teuthology.orchestra.run.smithi002.stdout:5536 trunc from 0xd437e02 to 0xc718d5b 2024-09-22T01:02:42.143 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:42.137+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:42.162 INFO:teuthology.orchestra.run.smithi002.stdout:5537 write 0x5eef13b thru 0x5ef0384 (0x124a bytes) 2024-09-22T01:02:42.178 INFO:teuthology.orchestra.run.smithi002.stdout:5538 write 0xca033a5 thru 0xca05910 (0x256c bytes) 2024-09-22T01:02:42.183 INFO:teuthology.orchestra.run.smithi002.stdout:5539 read 0x44e45e9 thru 0x44f2822 (0xe23a bytes) 2024-09-22T01:02:42.187 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:42.187 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:42.194 INFO:teuthology.orchestra.run.smithi002.stdout:5540 trunc from 0xca05911 to 0x50b3fcc 2024-09-22T01:02:42.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:42.218 INFO:teuthology.orchestra.run.smithi002.stdout:5541 write 0x27de21 thru 0x288648 (0xa828 bytes) 2024-09-22T01:02:42.227 INFO:teuthology.orchestra.run.smithi002.stdout:5542 read 0x229e1a8 thru 0x22aaf96 (0xcdef bytes) 2024-09-22T01:02:42.231 INFO:teuthology.orchestra.run.smithi002.stdout:5543 trunc from 0x50b3fcc to 0x66d985e 2024-09-22T01:02:42.239 INFO:teuthology.orchestra.run.smithi002.stdout:5544 write 0x4d355f5 thru 0x4d42c90 (0xd69c bytes) 2024-09-22T01:02:42.244 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:42.241+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:42.245 INFO:teuthology.orchestra.run.smithi002.stdout:5545 punch from 0x1c8648a to 0x1c90e3a, (0xa9b0 bytes) 2024-09-22T01:02:42.245 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.265 INFO:teuthology.orchestra.run.smithi002.stdout:5549 write 0xa12b487 thru 0xa130223 (0x4d9d bytes) 2024-09-22T01:02:42.270 INFO:teuthology.orchestra.run.smithi002.stdout:5550 write 0x3e2d24a thru 0x3e3650e (0x92c5 bytes) 2024-09-22T01:02:42.304 INFO:teuthology.orchestra.run.smithi002.stdout:5551 writesame 0xe7827c3 thru 0xe7920f4 data_size 0x2(0xf932 bytes) 2024-09-22T01:02:42.304 INFO:teuthology.orchestra.run.smithi002.stdout:5552 write 0xe0f8f74 thru 0xe0fe968 (0x59f5 bytes) 2024-09-22T01:02:42.312 INFO:teuthology.orchestra.run.smithi002.stdout:5553 writesame 0xb71a268 thru 0xb726d97 data_size 0x8(0xcb30 bytes) 2024-09-22T01:02:42.322 INFO:teuthology.orchestra.run.smithi002.stdout:5555 write 0xa2043b6 thru 0xa208d28 (0x4973 bytes) 2024-09-22T01:02:42.330 INFO:teuthology.orchestra.run.smithi002.stdout:5558 read 0x3643ee0 thru 0x3650cc7 (0xcde8 bytes) 2024-09-22T01:02:42.331 INFO:teuthology.orchestra.run.smithi002.stdout:5560 write 0x82b5388 thru 0x82ba841 (0x54ba bytes) 2024-09-22T01:02:42.345 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:42.341+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:42.347 INFO:teuthology.orchestra.run.smithi002.stdout:5561 trunc from 0xe7920f5 to 0xb37dddf 2024-09-22T01:02:42.373 INFO:teuthology.orchestra.run.smithi002.stdout:5564 read 0x5fd75c6 thru 0x5fe72ef (0xfd2a bytes) 2024-09-22T01:02:42.374 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:42.374 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:42.388 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:42.394 INFO:teuthology.orchestra.run.smithi002.stdout:5565 trunc from 0xb37dddf to 0x366bbe7 2024-09-22T01:02:42.394 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:42.421 INFO:teuthology.orchestra.run.smithi002.stdout:5566 punch from 0x274af2c to 0x274e54f, (0x3623 bytes) 2024-09-22T01:02:42.446 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:42.441+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:42.448 INFO:teuthology.orchestra.run.smithi002.stdout:5567 writesame 0xe9536a2 thru 0xe95ca4c data_size 0x1(0x93ab bytes) 2024-09-22T01:02:42.448 INFO:teuthology.orchestra.run.smithi002.stdout:5568 write 0xc76f811 thru 0xc775bee (0x63de bytes) 2024-09-22T01:02:42.454 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.454 INFO:teuthology.orchestra.run.smithi002.stdout:5570 read 0x3135d1f thru 0x3138f41 (0x3223 bytes) 2024-09-22T01:02:42.454 INFO:teuthology.orchestra.run.smithi002.stdout:5571 write 0xa029696 thru 0xa03308e (0x99f9 bytes) 2024-09-22T01:02:42.461 INFO:teuthology.orchestra.run.smithi002.stdout:5572 read 0x25a8bb3 thru 0x25b8273 (0xf6c1 bytes) 2024-09-22T01:02:42.461 INFO:teuthology.orchestra.run.smithi002.stdout:5573 write 0x347e1fd thru 0x3483a98 (0x589c bytes) 2024-09-22T01:02:42.469 INFO:teuthology.orchestra.run.smithi002.stdout:5574 write 0xac3abab thru 0xac48709 (0xdb5f bytes) 2024-09-22T01:02:42.475 INFO:teuthology.orchestra.run.smithi002.stdout:5576 write 0xce026c8 thru 0xce0a3ea (0x7d23 bytes) 2024-09-22T01:02:42.482 INFO:teuthology.orchestra.run.smithi002.stdout:5577 trunc from 0xe95ca4d to 0xdb1a64e 2024-09-22T01:02:42.492 INFO:teuthology.orchestra.run.smithi002.stdout:5578 writesame 0x7171f54 thru 0x717e318 data_size 0x1(0xc3c5 bytes) 2024-09-22T01:02:42.528 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:42.528 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:42.544 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:42.547 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:42.541+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:42.555 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:42.562 INFO:teuthology.orchestra.run.smithi002.stdout:5580 read 0x108cf43 thru 0x1092fa1 (0x605f bytes) 2024-09-22T01:02:42.562 INFO:teuthology.orchestra.run.smithi002.stdout:5581 punch from 0x596bcca to 0x597779b, (0xbad1 bytes) 2024-09-22T01:02:42.562 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.562 INFO:teuthology.orchestra.run.smithi002.stdout:5585 writesame 0x9869799 thru 0x9876f41 data_size 0x4d(0xd7a9 bytes) 2024-09-22T01:02:42.568 INFO:teuthology.orchestra.run.smithi002.stdout:5588 read 0x6679506 thru 0x6688896 (0xf391 bytes) 2024-09-22T01:02:42.569 INFO:teuthology.orchestra.run.smithi002.stdout:5589 read 0xd9c4b64 thru 0xd9ca929 (0x5dc6 bytes) 2024-09-22T01:02:42.569 INFO:teuthology.orchestra.run.smithi002.stdout:5590 read 0xb4c6f21 thru 0xb4d18c9 (0xa9a9 bytes) 2024-09-22T01:02:42.569 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.569 INFO:teuthology.orchestra.run.smithi002.stdout:5596 write 0x52e7c86 thru 0x52f29ee (0xad69 bytes) 2024-09-22T01:02:42.575 INFO:teuthology.orchestra.run.smithi002.stdout:5597 writesame 0x23c07f8 thru 0x23c6e4a data_size 0x41(0x6653 bytes) 2024-09-22T01:02:42.576 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:42.590 INFO:teuthology.orchestra.run.smithi002.stdout:5598 writesame 0xe30e73a thru 0xe30f8a7 data_size 0x2(0x116e bytes) 2024-09-22T01:02:42.590 INFO:teuthology.orchestra.run.smithi002.stdout:5599 punch from 0x8738275 to 0x87400bf, (0x7e4a bytes) 2024-09-22T01:02:42.590 INFO:teuthology.orchestra.run.smithi002.stdout:5600 writesame 0x7ad6f01 thru 0x7ae5825 data_size 0x17(0xe925 bytes) 2024-09-22T01:02:42.597 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:42.597 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:42.598 INFO:teuthology.orchestra.run.smithi002.stdout:5601 read 0x192137a thru 0x192381c (0x24a3 bytes) 2024-09-22T01:02:42.598 INFO:teuthology.orchestra.run.smithi002.stdout:5604 read 0xc0f20d7 thru 0xc0f871f (0x6649 bytes) 2024-09-22T01:02:42.598 INFO:teuthology.orchestra.run.smithi002.stdout:5606 read 0x98ec03e thru 0x98ec512 (0x4d5 bytes) 2024-09-22T01:02:42.607 INFO:teuthology.orchestra.run.smithi002.stdout:5607 trunc from 0xe30f8a8 to 0x7ae7463 2024-09-22T01:02:42.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:42.618 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:42.627 INFO:teuthology.orchestra.run.smithi002.stdout:5608 punch from 0x15d4b46 to 0x15d5d37, (0x11f1 bytes) 2024-09-22T01:02:42.627 INFO:teuthology.orchestra.run.smithi002.stdout:5609 punch from 0x23b8a64 to 0x23c538b, (0xc927 bytes) 2024-09-22T01:02:42.627 INFO:teuthology.orchestra.run.smithi002.stdout:5611 read 0x4b0b2b2 thru 0x4b111be (0x5f0d bytes) 2024-09-22T01:02:42.627 INFO:teuthology.orchestra.run.smithi002.stdout:5612 punch from 0x1cb97c0 to 0x1cc83bb, (0xebfb bytes) 2024-09-22T01:02:42.639 INFO:teuthology.orchestra.run.smithi002.stdout:5614 trunc from 0x7ae7463 to 0x3218ef 2024-09-22T01:02:42.640 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:42.647 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:42.641+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:42.664 INFO:teuthology.orchestra.run.smithi002.stdout:5615 punch from 0x312957 to 0x3205b9, (0xdc62 bytes) 2024-09-22T01:02:42.664 INFO:teuthology.orchestra.run.smithi002.stdout:5616 read 0x35c3c thru 0x42588 (0xc94d bytes) 2024-09-22T01:02:42.665 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:42.665 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:42.686 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:42.686 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:42.689 INFO:teuthology.orchestra.run.smithi002.stdout:5617 write 0xa91983a thru 0xa928da8 (0xf56f bytes) 2024-09-22T01:02:42.696 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.696 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.696 INFO:teuthology.orchestra.run.smithi002.stdout:5623 write 0x2fd2d6a thru 0x2fd3620 (0x8b7 bytes) 2024-09-22T01:02:42.700 INFO:teuthology.orchestra.run.smithi002.stdout:5624 writesame 0x52b67eb thru 0x52c0cf0 data_size 0x9(0xa506 bytes) 2024-09-22T01:02:42.707 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:42.708 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:42.710 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:42.720 INFO:teuthology.orchestra.run.smithi002.stdout:5625 write 0xcb4972a thru 0xcb55524 (0xbdfb bytes) 2024-09-22T01:02:42.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:42.743 INFO:teuthology.orchestra.run.smithi002.stdout:5626 trunc from 0xcb55525 to 0x423089 2024-09-22T01:02:42.762 INFO:teuthology.orchestra.run.smithi002.stdout:5627 read 0x12b38 thru 0x1e847 (0xbd10 bytes) 2024-09-22T01:02:42.762 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:42.761+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:42.763 INFO:teuthology.orchestra.run.smithi002.stdout:5628 read 0x343bd7 thru 0x352cb1 (0xf0db bytes) 2024-09-22T01:02:42.767 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.5 to 1 (10000) 2024-09-22T01:02:42.775 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:02:42.790 INFO:teuthology.orchestra.run.smithi002.stdout:5629 write 0xa47b151 thru 0xa47bc7c (0xb2c bytes) 2024-09-22T01:02:42.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-22T01:02:42.797 INFO:teuthology.orchestra.run.smithi002.stdout:5630 write 0x523687d thru 0x5240e04 (0xa588 bytes) 2024-09-22T01:02:42.805 INFO:teuthology.orchestra.run.smithi002.stdout:5631 punch from 0x8fab328 to 0x8fb2c5f, (0x7937 bytes) 2024-09-22T01:02:42.805 INFO:teuthology.orchestra.run.smithi002.stdout:5632 write 0x7100a18 thru 0x71072d6 (0x68bf bytes) 2024-09-22T01:02:42.811 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.822 INFO:teuthology.orchestra.run.smithi002.stdout:5635 trunc from 0xa47bc7d to 0x3b82381 2024-09-22T01:02:42.839 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.841 INFO:teuthology.orchestra.run.smithi002.stdout:5639 trunc from 0x3b82381 to 0x5090975 2024-09-22T01:02:42.849 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:42.845+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:42.864 INFO:teuthology.orchestra.run.smithi002.stdout:5640 write 0x8e604f9 thru 0x8e64051 (0x3b59 bytes) 2024-09-22T01:02:42.870 INFO:teuthology.orchestra.run.smithi002.stdout:5641 read 0x1e8cb31 thru 0x1e91c39 (0x5109 bytes) 2024-09-22T01:02:42.870 INFO:teuthology.orchestra.run.smithi002.stdout:5645 write 0x704629e thru 0x704910d (0x2e70 bytes) 2024-09-22T01:02:42.876 INFO:teuthology.orchestra.run.smithi002.stdout:5646 read 0x8ccd46f thru 0x8cd1b42 (0x46d4 bytes) 2024-09-22T01:02:42.876 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.876 INFO:teuthology.orchestra.run.smithi002.stdout:5650 write 0xf8176d thru 0xf83a80 (0x2314 bytes) 2024-09-22T01:02:42.882 INFO:teuthology.orchestra.run.smithi002.stdout:5651 read 0x41cd1ab thru 0x41cffdf (0x2e35 bytes) 2024-09-22T01:02:42.882 INFO:teuthology.orchestra.run.smithi002.stdout:5653 punch from 0x8a536f6 to 0x8a6242a, (0xed34 bytes) 2024-09-22T01:02:42.883 INFO:teuthology.orchestra.run.smithi002.stdout:5654 write 0x6a861a0 thru 0x6a8fe55 (0x9cb6 bytes) 2024-09-22T01:02:42.889 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:42.889 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:42.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:42.889 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:42.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.889 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.389002138999999, 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:02:42.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:02:42.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.388974296000001, 2024-09-22T01:02:42.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:02:42.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:02:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.38894208, 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.899 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.388920341999999, 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:02:42.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.388900687000003, 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.908 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:02:42.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.388869673000002, 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.912 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:42.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92263 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 20971520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.916 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433510505000001, 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049994333000000002, 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92263 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454536+0000", 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454537+0000", 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454541+0000", 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7129999999999999e-06 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454551+0000", 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0448000000000001e-05 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454558+0000", 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.323e-06 2024-09-22T01:02:42.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504317+0000", 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049758335000000001 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504408+0000", 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1508999999999994e-05 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504530+0000", 2024-09-22T01:02:42.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122168 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433465462000001, 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:02:42.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:42.925 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433425827000001, 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:42.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433338386000003, 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:02:42.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433298166999997, 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.938 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433262274999997, 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:02:42.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433214626999998, 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:02:42.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433179858000003, 2024-09-22T01:02:42.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:02:42.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.951 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433146665000002, 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:02:42.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433102451000003, 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:02:42.956 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:02:42.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433060261999998, 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:02:42.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.965 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.433026150000003, 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:02:42.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:02:42.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.432993222999997, 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:02:42.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.432958454999998, 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:02:42.976 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:02:42.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:42.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:42.981 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:42.981 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:42.981 INFO:teuthology.orchestra.run.smithi002.stdout:5656 writesame 0x1fffed1 thru 0x2000a26 data_size 0x1(0xb56 bytes) 2024-09-22T01:02:42.982 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:42.983 INFO:teuthology.orchestra.run.smithi002.stdout:5658 writesame 0xebdc222 thru 0xebeb27d data_size 0x2(0xf05c bytes) 2024-09-22T01:02:42.983 INFO:teuthology.orchestra.run.smithi002.stdout:5659 trunc from 0xebeb27e to 0x5e670d4 2024-09-22T01:02:42.983 INFO:teuthology.orchestra.run.smithi002.stdout:5660 read 0xdb3482 thru 0xdb6c7c (0x37fb bytes) 2024-09-22T01:02:42.984 INFO:teuthology.orchestra.run.smithi002.stdout:5661 read 0xed209f thru 0xed70f8 (0x505a bytes) 2024-09-22T01:02:42.984 INFO:teuthology.orchestra.run.smithi002.stdout:5662 punch from 0x5dbf888 to 0x5dc8bac, (0x9324 bytes) 2024-09-22T01:02:42.984 INFO:teuthology.orchestra.run.smithi002.stdout:5664 write 0x9bf6121 thru 0x9bfe1e4 (0x80c4 bytes) 2024-09-22T01:02:42.984 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.984 INFO:teuthology.orchestra.run.smithi002.stdout:5666 trunc from 0x9bfe1e5 to 0xd9a8ea1 2024-09-22T01:02:42.984 INFO:teuthology.orchestra.run.smithi002.stdout:5668 read 0x7a48123 thru 0x7a56b45 (0xea23 bytes) 2024-09-22T01:02:42.984 INFO:teuthology.orchestra.run.smithi002.stdout:5671 writesame 0x40e9401 thru 0x40f5559 data_size 0x7(0xc159 bytes) 2024-09-22T01:02:42.991 INFO:teuthology.orchestra.run.smithi002.stdout:5673 punch from 0x5adafa6 to 0x5ae0d82, (0x5ddc bytes) 2024-09-22T01:02:42.991 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.991 INFO:teuthology.orchestra.run.smithi002.stdout:5675 read 0x6e9952a thru 0x6e99bf3 (0x6ca bytes) 2024-09-22T01:02:42.991 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:42.992 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:42.989+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:42.993 INFO:teuthology.orchestra.run.smithi002.stdout:5678 trunc from 0xd9a8ea1 to 0xec7f523 2024-09-22T01:02:43.001 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.001 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.001 INFO:teuthology.orchestra.run.smithi002.stdout:5681 writesame 0xa069af4 thru 0xa06a0c3 data_size 0x18(0x5d0 bytes) 2024-09-22T01:02:43.024 INFO:teuthology.orchestra.run.smithi002.stdout:5682 trunc from 0xec7f523 to 0x28087c3 2024-09-22T01:02:43.042 INFO:teuthology.orchestra.run.smithi002.stdout:5683 writesame 0xf835e5 thru 0xf90ce1 data_size 0x1(0xd6fd bytes) 2024-09-22T01:02:43.072 INFO:teuthology.orchestra.run.smithi002.stdout:5684 read 0x2102668 thru 0x2104971 (0x230a bytes) 2024-09-22T01:02:43.086 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:43.081+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:43.088 INFO:teuthology.orchestra.run.smithi002.stdout:5685 trunc from 0x28087c3 to 0xc64c5b8 2024-09-22T01:02:43.098 INFO:teuthology.orchestra.run.smithi002.stdout:5686 read 0x5786c3c thru 0x5790b88 (0x9f4d bytes) 2024-09-22T01:02:43.098 INFO:teuthology.orchestra.run.smithi002.stdout:5687 read 0x320dbf4 thru 0x321983d (0xbc4a bytes) 2024-09-22T01:02:43.098 INFO:teuthology.orchestra.run.smithi002.stdout:5688 read 0x78f9045 thru 0x7902362 (0x931e bytes) 2024-09-22T01:02:43.098 INFO:teuthology.orchestra.run.smithi002.stdout:5689 write 0xab53caa thru 0xab5e517 (0xa86e bytes) 2024-09-22T01:02:43.104 INFO:teuthology.orchestra.run.smithi002.stdout:5690 write 0x2f553cf thru 0x2f637b1 (0xe3e3 bytes) 2024-09-22T01:02:43.111 INFO:teuthology.orchestra.run.smithi002.stdout:5692 write 0xac4ffdb thru 0xac5fa8a (0xfab0 bytes) 2024-09-22T01:02:43.117 INFO:teuthology.orchestra.run.smithi002.stdout:5693 punch from 0x3ce7220 to 0x3cefecb, (0x8cab bytes) 2024-09-22T01:02:43.118 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.118 INFO:teuthology.orchestra.run.smithi002.stdout:5695 punch from 0x4e9bd57 to 0x4ea9646, (0xd8ef bytes) 2024-09-22T01:02:43.118 INFO:teuthology.orchestra.run.smithi002.stdout:5696 punch from 0x37d1065 to 0x37d35b6, (0x2551 bytes) 2024-09-22T01:02:43.118 INFO:teuthology.orchestra.run.smithi002.stdout:5697 writesame 0x3272ffd thru 0x32824f5 data_size 0x77(0xf4f9 bytes) 2024-09-22T01:02:43.124 INFO:teuthology.orchestra.run.smithi002.stdout:5698 read 0x37fd96b thru 0x38078e7 (0x9f7d bytes) 2024-09-22T01:02:43.130 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:43.130 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:43.130 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:43.130 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:43.130 INFO:teuthology.orchestra.run.smithi002.stdout:5699 trunc from 0xc64c5b8 to 0x883683c 2024-09-22T01:02:43.147 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:43.149 INFO:teuthology.orchestra.run.smithi002.stdout:5700 read 0x67cd7aa thru 0x67cf1af (0x1a06 bytes) 2024-09-22T01:02:43.149 INFO:teuthology.orchestra.run.smithi002.stdout:5701 read 0x14d963a thru 0x14e73c3 (0xdd8a bytes) 2024-09-22T01:02:43.151 INFO:teuthology.orchestra.run.smithi002.stdout:5702 trunc from 0x883683c to 0x6eae62b 2024-09-22T01:02:43.165 INFO:teuthology.orchestra.run.smithi002.stdout:5703 trunc from 0x6eae62b to 0xb9f93a9 2024-09-22T01:02:43.173 INFO:teuthology.orchestra.run.smithi002.stdout:5704 read 0x6667de7 thru 0x6670016 (0x8230 bytes) 2024-09-22T01:02:43.173 INFO:teuthology.orchestra.run.smithi002.stdout:5706 punch from 0x10ac6f to 0x10ee7e, (0x420f bytes) 2024-09-22T01:02:43.173 INFO:teuthology.orchestra.run.smithi002.stdout:5707 write 0x524dc57 thru 0x525c0e8 (0xe492 bytes) 2024-09-22T01:02:43.177 INFO:teuthology.orchestra.run.smithi002.stdout:5708 punch from 0xad0665f to 0xad0ab17, (0x44b8 bytes) 2024-09-22T01:02:43.185 INFO:teuthology.orchestra.run.smithi002.stdout:5709 writesame 0xc09adcf thru 0xc0a159e data_size 0x58(0x67d0 bytes) 2024-09-22T01:02:43.185 INFO:teuthology.orchestra.run.smithi002.stdout:5710 punch from 0x72116db to 0x721da03, (0xc328 bytes) 2024-09-22T01:02:43.185 INFO:teuthology.orchestra.run.smithi002.stdout:5711 write 0x42c3127 thru 0x42d24ab (0xf385 bytes) 2024-09-22T01:02:43.186 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:43.185+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:43.191 INFO:teuthology.orchestra.run.smithi002.stdout:5712 read 0x85ab30b thru 0x85b3a0f (0x8705 bytes) 2024-09-22T01:02:43.191 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.191 INFO:teuthology.orchestra.run.smithi002.stdout:5718 read 0x46d2e49 thru 0x46d8f8a (0x6142 bytes) 2024-09-22T01:02:43.191 INFO:teuthology.orchestra.run.smithi002.stdout:5719 read 0xa9418f1 thru 0xa943eb6 (0x25c6 bytes) 2024-09-22T01:02:43.206 INFO:teuthology.orchestra.run.smithi002.stdout:5720 trunc from 0xc0a159f to 0x1497fd 2024-09-22T01:02:43.253 INFO:teuthology.orchestra.run.smithi002.stdout:5723 write 0x7da18c5 thru 0x7db1584 (0xfcc0 bytes) 2024-09-22T01:02:43.258 INFO:teuthology.orchestra.run.smithi002.stdout:5724 punch from 0x67dd388 to 0x67e8da4, (0xba1c bytes) 2024-09-22T01:02:43.268 INFO:teuthology.orchestra.run.smithi002.stdout:5725 write 0x9b90180 thru 0x9b914ec (0x136d bytes) 2024-09-22T01:02:43.277 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:43.277 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:43.278 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:43.278 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:43.278 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:43.279 INFO:teuthology.orchestra.run.smithi002.stdout:5726 trunc from 0x9b914ed to 0x5ede94c 2024-09-22T01:02:43.287 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:43.281+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:43.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:43.313 INFO:teuthology.orchestra.run.smithi002.stdout:5727 write 0xe384d8b thru 0xe3927be (0xda34 bytes) 2024-09-22T01:02:43.318 INFO:teuthology.orchestra.run.smithi002.stdout:5728 write 0xb4f4208 thru 0xb4fa328 (0x6121 bytes) 2024-09-22T01:02:43.323 INFO:teuthology.orchestra.run.smithi002.stdout:5729 write 0xc12df2e thru 0xc12e1e2 (0x2b5 bytes) 2024-09-22T01:02:43.329 INFO:teuthology.orchestra.run.smithi002.stdout:5730 read 0x5c452b2 thru 0x5c4e6a2 (0x93f1 bytes) 2024-09-22T01:02:43.329 INFO:teuthology.orchestra.run.smithi002.stdout:5731 write 0xd84d990 thru 0xd85004b (0x26bc bytes) 2024-09-22T01:02:43.335 INFO:teuthology.orchestra.run.smithi002.stdout:5733 punch from 0x6de345b to 0x6de4f37, (0x1adc bytes) 2024-09-22T01:02:43.335 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.336 INFO:teuthology.orchestra.run.smithi002.stdout:5738 punch from 0x8660d69 to 0x8661f7e, (0x1215 bytes) 2024-09-22T01:02:43.336 INFO:teuthology.orchestra.run.smithi002.stdout:5739 read 0xa5165cd thru 0xa51b487 (0x4ebb bytes) 2024-09-22T01:02:43.336 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.336 INFO:teuthology.orchestra.run.smithi002.stdout:5741 write 0xaa27f6e thru 0xaa2c7a4 (0x4837 bytes) 2024-09-22T01:02:43.344 INFO:teuthology.orchestra.run.smithi002.stdout:5743 trunc from 0xe3927bf to 0xbd289da 2024-09-22T01:02:43.361 INFO:teuthology.orchestra.run.smithi002.stdout:5744 write 0x454ffb2 thru 0x455a254 (0xa2a3 bytes) 2024-09-22T01:02:43.366 INFO:teuthology.orchestra.run.smithi002.stdout:5745 writesame 0x3d6b9a5 thru 0x3d6d3d2 data_size 0x3(0x1a2e bytes) 2024-09-22T01:02:43.385 INFO:teuthology.orchestra.run.smithi002.stdout:5747 write 0xde75c22 thru 0xde7dde5 (0x81c4 bytes) 2024-09-22T01:02:43.388 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:43.381+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:43.398 INFO:teuthology.orchestra.run.smithi002.stdout:5749 trunc from 0xde7dde6 to 0x897f0ee 2024-09-22T01:02:43.434 INFO:teuthology.orchestra.run.smithi002.stdout:5750 write 0xb0e1672 thru 0xb0e8403 (0x6d92 bytes) 2024-09-22T01:02:43.442 INFO:teuthology.orchestra.run.smithi002.stdout:5752 read 0x607e1b1 thru 0x607ee3a (0xc8a bytes) 2024-09-22T01:02:43.442 INFO:teuthology.orchestra.run.smithi002.stdout:5753 read 0x59d0726 thru 0x59d94fa (0x8dd5 bytes) 2024-09-22T01:02:43.445 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.3648841, 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:02:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364865915, 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:02:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:02:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364848127, 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.36482956899999, 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:43.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:02:43.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.36481151700001, 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364794168, 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:02:43.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364772215, 2024-09-22T01:02:43.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.475 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.36475468, 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364730894, 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:02:43.480 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:02:43.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364713032, 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:02:43.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:02:43.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364685795, 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:02:43.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:02:43.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.36466709, 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.494 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:02:43.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:02:43.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:02:43.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364643593, 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:02:43.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:02:43.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364619624, 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:02:43.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.364600409, 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:02:43.507 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:02:43.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:02:43.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.363595859, 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:02:43.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:02:43.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 108.361089906, 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.520 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.645631745999999, 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:02:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:140499 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:02:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3869555760000001, 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98072477000000002, 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140499 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057286+0000", 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057297+0000", 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026e-05 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057320+0000", 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3391999999999999e-05 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057331+0000", 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.075e-05 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057403+0000", 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057411+0000", 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2749999999999995e-06 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.037974+0000", 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98056295800000004 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.038008+0000", 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3844999999999999e-05 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:144964 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6627000169999997, 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95820099800000003, 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144964 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781542+0000", 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4899999999999999e-06 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781553+0000", 2024-09-22T01:02:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1687e-05 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781583+0000", 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0173999999999999e-05 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781597+0000", 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3172e-05 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781681+0000", 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4810999999999998e-05 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781692+0000", 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0662000000000001e-05 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739463+0000", 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95777067000000005 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739603+0000", 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013200000000001 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739740+0000", 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001372 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:43.536 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:43.537 INFO:teuthology.orchestra.run.smithi002.stdout:5754 trunc from 0xb0e8404 to 0x8d87781 2024-09-22T01:02:43.538 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:43.539 INFO:teuthology.orchestra.run.smithi002.stdout:5756 writesame 0x33d0f37 thru 0x33d27a2 data_size 0x6(0x186c bytes) 2024-09-22T01:02:43.539 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.539 INFO:teuthology.orchestra.run.smithi002.stdout:5758 write 0x74fa07b thru 0x74fc7d8 (0x275e bytes) 2024-09-22T01:02:43.539 INFO:teuthology.orchestra.run.smithi002.stdout:5760 read 0x87c70cd thru 0x87ca262 (0x3196 bytes) 2024-09-22T01:02:43.539 INFO:teuthology.orchestra.run.smithi002.stdout:5761 writesame 0xad8f6a9 thru 0xad9bbf2 data_size 0x1(0xc54a bytes) 2024-09-22T01:02:43.539 INFO:teuthology.orchestra.run.smithi002.stdout:5764 write 0xc9222da thru 0xc9259a2 (0x36c9 bytes) 2024-09-22T01:02:43.540 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:43.533+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:43.540 INFO:teuthology.orchestra.run.smithi002.stdout:5765 writesame 0xe6eaf97 thru 0xe6ed086 data_size 0x22(0x20f0 bytes) 2024-09-22T01:02:43.540 INFO:teuthology.orchestra.run.smithi002.stdout:5766 read 0x95cd0ec thru 0x95cfb69 (0x2a7e bytes) 2024-09-22T01:02:43.540 INFO:teuthology.orchestra.run.smithi002.stdout:5767 write 0x96d968d thru 0x96ddedf (0x4853 bytes) 2024-09-22T01:02:43.546 INFO:teuthology.orchestra.run.smithi002.stdout:5768 write 0x5a862b8 thru 0x5a8c4b8 (0x6201 bytes) 2024-09-22T01:02:43.554 INFO:teuthology.orchestra.run.smithi002.stdout:5769 read 0x1beade2 thru 0x1bf5c4c (0xae6b bytes) 2024-09-22T01:02:43.554 INFO:teuthology.orchestra.run.smithi002.stdout:5770 write 0x3f052da thru 0x3f117a5 (0xc4cc bytes) 2024-09-22T01:02:43.560 INFO:teuthology.orchestra.run.smithi002.stdout:5771 punch from 0x715a852 to 0x715ff86, (0x5734 bytes) 2024-09-22T01:02:43.560 INFO:teuthology.orchestra.run.smithi002.stdout:5772 read 0x50bc788 thru 0x50c84c3 (0xbd3c bytes) 2024-09-22T01:02:43.561 INFO:teuthology.orchestra.run.smithi002.stdout:5775 trunc from 0xe6ed087 to 0xddbe863 2024-09-22T01:02:43.571 INFO:teuthology.orchestra.run.smithi002.stdout:5776 punch from 0xa24fd9d to 0xa25dec0, (0xe123 bytes) 2024-09-22T01:02:43.571 INFO:teuthology.orchestra.run.smithi002.stdout:5777 read 0x7ba4031 thru 0x7bafc96 (0xbc66 bytes) 2024-09-22T01:02:43.571 INFO:teuthology.orchestra.run.smithi002.stdout:5778 read 0xdb33943 thru 0xdb3a885 (0x6f43 bytes) 2024-09-22T01:02:43.571 INFO:teuthology.orchestra.run.smithi002.stdout:5779 write 0x7dd6143 thru 0x7dd9acb (0x3989 bytes) 2024-09-22T01:02:43.587 INFO:teuthology.orchestra.run.smithi002.stdout:5780 write 0xe5b0f58 thru 0xe5b2d08 (0x1db1 bytes) 2024-09-22T01:02:43.592 INFO:teuthology.orchestra.run.smithi002.stdout:5781 read 0xda9f50e thru 0xdaab045 (0xbb38 bytes) 2024-09-22T01:02:43.592 INFO:teuthology.orchestra.run.smithi002.stdout:5783 read 0xcb526a thru 0xcbbc2f (0x69c6 bytes) 2024-09-22T01:02:43.597 INFO:teuthology.orchestra.run.smithi002.stdout:5785 trunc from 0xe5b2d09 to 0xbb5204c 2024-09-22T01:02:43.617 INFO:teuthology.orchestra.run.smithi002.stdout:5786 read 0x29d1a62 thru 0x29de482 (0xca21 bytes) 2024-09-22T01:02:43.620 INFO:teuthology.orchestra.run.smithi002.stdout:5787 trunc from 0xbb5204c to 0xa724b52 2024-09-22T01:02:43.631 INFO:teuthology.orchestra.run.smithi002.stdout:5788 read 0x2d99e4b thru 0x2da69a3 (0xcb59 bytes) 2024-09-22T01:02:43.631 INFO:teuthology.orchestra.run.smithi002.stdout:5789 read 0x825a8f3 thru 0x8262417 (0x7b25 bytes) 2024-09-22T01:02:43.631 INFO:teuthology.orchestra.run.smithi002.stdout:5790 read 0x522ca39 thru 0x5238ec5 (0xc48d bytes) 2024-09-22T01:02:43.637 INFO:teuthology.orchestra.run.smithi002.stdout:5792 trunc from 0xa724b52 to 0xe5b19dc 2024-09-22T01:02:43.638 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:43.633+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:43.650 INFO:teuthology.orchestra.run.smithi002.stdout:5795 write 0x106077d thru 0x106bb9f (0xb423 bytes) 2024-09-22T01:02:43.658 INFO:teuthology.orchestra.run.smithi002.stdout:5797 punch from 0xb0cacb0 to 0xb0d5776, (0xaac6 bytes) 2024-09-22T01:02:43.658 INFO:teuthology.orchestra.run.smithi002.stdout:5798 punch from 0x57844b1 to 0x5785236, (0xd85 bytes) 2024-09-22T01:02:43.658 INFO:teuthology.orchestra.run.smithi002.stdout:5801 read 0xa68a8ac thru 0xa68ec94 (0x43e9 bytes) 2024-09-22T01:02:43.662 INFO:teuthology.orchestra.run.smithi002.stdout:5804 trunc from 0xe5b19dc to 0xb6da626 2024-09-22T01:02:43.674 INFO:teuthology.orchestra.run.smithi002.stdout:5805 punch from 0x43c8d86 to 0x43d67af, (0xda29 bytes) 2024-09-22T01:02:43.674 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.674 INFO:teuthology.orchestra.run.smithi002.stdout:5807 read 0x392fc17 thru 0x39365ec (0x69d6 bytes) 2024-09-22T01:02:43.675 INFO:teuthology.orchestra.run.smithi002.stdout:5808 write 0xa7745c7 thru 0xa783a46 (0xf480 bytes) 2024-09-22T01:02:43.680 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:43.680 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:43.683 INFO:teuthology.orchestra.run.smithi002.stdout:5809 punch from 0xac19977 to 0xac1bc05, (0x228e bytes) 2024-09-22T01:02:43.683 INFO:teuthology.orchestra.run.smithi002.stdout:5810 writesame 0x26c183e thru 0x26c2394 data_size 0x1(0xb57 bytes) 2024-09-22T01:02:43.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:43.700 INFO:teuthology.orchestra.run.smithi002.stdout:5812 trunc from 0xb6da626 to 0x499863e 2024-09-22T01:02:43.718 INFO:teuthology.orchestra.run.smithi002.stdout:5813 read 0x9b81e7 thru 0x9c6cd2 (0xeaec bytes) 2024-09-22T01:02:43.727 INFO:teuthology.orchestra.run.smithi002.stdout:5815 write 0x5439305 thru 0x543960e (0x30a bytes) 2024-09-22T01:02:43.732 INFO:teuthology.orchestra.run.smithi002.stdout:5820 write 0x1d5b740 thru 0x1d62bf8 (0x74b9 bytes) 2024-09-22T01:02:43.739 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:43.733+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:43.739 INFO:teuthology.orchestra.run.smithi002.stdout:5821 write 0x2c442c9 thru 0x2c4dadc (0x9814 bytes) 2024-09-22T01:02:43.745 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.758 INFO:teuthology.orchestra.run.smithi002.stdout:5823 write 0xaca9953 thru 0xaca9d1d (0x3cb bytes) 2024-09-22T01:02:43.764 INFO:teuthology.orchestra.run.smithi002.stdout:5826 writesame 0x527e53c thru 0x528050e data_size 0x1(0x1fd3 bytes) 2024-09-22T01:02:43.765 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.6 to 1 (10000) 2024-09-22T01:02:43.768 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:02:43.782 INFO:teuthology.orchestra.run.smithi002.stdout:5827 read 0x3a609d2 thru 0x3a620d7 (0x1706 bytes) 2024-09-22T01:02:43.782 INFO:teuthology.orchestra.run.smithi002.stdout:5829 read 0x2802db9 thru 0x280b0ec (0x8334 bytes) 2024-09-22T01:02:43.782 INFO:teuthology.orchestra.run.smithi002.stdout:5831 punch from 0x31e8ea5 to 0x31ed983, (0x4ade bytes) 2024-09-22T01:02:43.782 INFO:teuthology.orchestra.run.smithi002.stdout:5832 write 0x5a4d860 thru 0x5a59d17 (0xc4b8 bytes) 2024-09-22T01:02:43.801 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-22T01:02:43.802 INFO:teuthology.orchestra.run.smithi002.stdout:5833 write 0xb803f4e thru 0xb80d163 (0x9216 bytes) 2024-09-22T01:02:43.808 INFO:teuthology.orchestra.run.smithi002.stdout:5834 read 0x68741aa thru 0x687e8ac (0xa703 bytes) 2024-09-22T01:02:43.808 INFO:teuthology.orchestra.run.smithi002.stdout:5835 read 0x5400d31 thru 0x540bf83 (0xb253 bytes) 2024-09-22T01:02:43.817 INFO:teuthology.orchestra.run.smithi002.stdout:5836 writesame 0xbe656e2 thru 0xbe6cb11 data_size 0x4(0x7430 bytes) 2024-09-22T01:02:43.818 INFO:teuthology.orchestra.run.smithi002.stdout:5837 read 0x12ddf94 thru 0x12e51db (0x7248 bytes) 2024-09-22T01:02:43.818 INFO:teuthology.orchestra.run.smithi002.stdout:5838 read 0x24c0372 thru 0x24ce2a7 (0xdf36 bytes) 2024-09-22T01:02:43.818 INFO:teuthology.orchestra.run.smithi002.stdout:5839 write 0x91bb356 thru 0x91c0890 (0x553b bytes) 2024-09-22T01:02:43.825 INFO:teuthology.orchestra.run.smithi002.stdout:5841 write 0x88ce7a7 thru 0x88d6caf (0x8509 bytes) 2024-09-22T01:02:43.830 INFO:teuthology.orchestra.run.smithi002.stdout:5842 punch from 0x1ab56aa to 0x1ab7c88, (0x25de bytes) 2024-09-22T01:02:43.830 INFO:teuthology.orchestra.run.smithi002.stdout:5843 write 0xa9d6ffd thru 0xa9da1e6 (0x31ea bytes) 2024-09-22T01:02:43.836 INFO:teuthology.orchestra.run.smithi002.stdout:5844 read 0x5a888b thru 0x5ae166 (0x58dc bytes) 2024-09-22T01:02:43.836 INFO:teuthology.orchestra.run.smithi002.stdout:5845 punch from 0x1c4fd4a to 0x1c52ba5, (0x2e5b bytes) 2024-09-22T01:02:43.836 INFO:teuthology.orchestra.run.smithi002.stdout:5846 writesame 0x8ee06c5 thru 0x8ee7673 data_size 0x1(0x6faf bytes) 2024-09-22T01:02:43.840 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:43.833+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:43.842 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:43.842 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:43.842 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:43.842 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:43.843 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:43.857 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:43.885 INFO:teuthology.orchestra.run.smithi002.stdout:5847 punch from 0x89a097 to 0x8a6788, (0xc6f1 bytes) 2024-09-22T01:02:43.885 INFO:teuthology.orchestra.run.smithi002.stdout:5848 punch from 0x6aec144 to 0x6afae76, (0xed32 bytes) 2024-09-22T01:02:43.893 INFO:teuthology.orchestra.run.smithi002.stdout:5850 write 0xbe73b52 thru 0xbe7865c (0x4b0b bytes) 2024-09-22T01:02:43.900 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.900 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.900 INFO:teuthology.orchestra.run.smithi002.stdout:5853 read 0x9e76ae9 thru 0x9e807af (0x9cc7 bytes) 2024-09-22T01:02:43.901 INFO:teuthology.orchestra.run.smithi002.stdout:5854 punch from 0xdb98f8 to 0xdba27d, (0x985 bytes) 2024-09-22T01:02:43.901 INFO:teuthology.orchestra.run.smithi002.stdout:5855 punch from 0xa861595 to 0xa862627, (0x1092 bytes) 2024-09-22T01:02:43.901 INFO:teuthology.orchestra.run.smithi002.stdout:5856 punch from 0xb0508d6 to 0xb05c09f, (0xb7c9 bytes) 2024-09-22T01:02:43.901 INFO:teuthology.orchestra.run.smithi002.stdout:5857 writesame 0x3d11a4c thru 0x3d170ef data_size 0xa(0x56a4 bytes) 2024-09-22T01:02:43.905 INFO:teuthology.orchestra.run.smithi002.stdout:5860 writesame 0x68e69ac thru 0x68ebd1d data_size 0xb(0x5372 bytes) 2024-09-22T01:02:43.912 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.912 INFO:teuthology.orchestra.run.smithi002.stdout:5865 punch from 0x4b232a0 to 0x4b24a5a, (0x17ba bytes) 2024-09-22T01:02:43.912 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.912 INFO:teuthology.orchestra.run.smithi002.stdout:5867 read 0x8f9f2db thru 0x8fae6e8 (0xf40e bytes) 2024-09-22T01:02:43.913 INFO:teuthology.orchestra.run.smithi002.stdout:5868 punch from 0xb693907 to 0xb69e791, (0xae8a bytes) 2024-09-22T01:02:43.913 INFO:teuthology.orchestra.run.smithi002.stdout:5869 read 0x97fe81a thru 0x9804560 (0x5d47 bytes) 2024-09-22T01:02:43.924 INFO:teuthology.orchestra.run.smithi002.stdout:5870 write 0xe599627 thru 0xe5a5227 (0xbc01 bytes) 2024-09-22T01:02:43.931 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.931 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.931 INFO:teuthology.orchestra.run.smithi002.stdout:5873 write 0x3eff9db thru 0x3f0a47f (0xaaa5 bytes) 2024-09-22T01:02:43.937 INFO:teuthology.orchestra.run.smithi002.stdout:5874 write 0x1c5ec3b thru 0x1c67d94 (0x915a bytes) 2024-09-22T01:02:43.940 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:43.933+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:43.945 INFO:teuthology.orchestra.run.smithi002.stdout:5875 punch from 0x92f7bd4 to 0x92fee9d, (0x72c9 bytes) 2024-09-22T01:02:43.945 INFO:teuthology.orchestra.run.smithi002.stdout:5877 punch from 0x1915114 to 0x19244d3, (0xf3bf bytes) 2024-09-22T01:02:43.945 INFO:teuthology.orchestra.run.smithi002.stdout:5878 writesame 0x5208f3e thru 0x52187c1 data_size 0xa(0xf884 bytes) 2024-09-22T01:02:43.950 INFO:teuthology.orchestra.run.smithi002.stdout:5879 read 0x8d0123e thru 0x8d07788 (0x654b bytes) 2024-09-22T01:02:43.951 INFO:teuthology.orchestra.run.smithi002.stdout:5880 write 0x96185e2 thru 0x962754c (0xef6b bytes) 2024-09-22T01:02:43.957 INFO:teuthology.orchestra.run.smithi002.stdout:5881 write 0x30d675e thru 0x30d75c3 (0xe66 bytes) 2024-09-22T01:02:43.962 INFO:teuthology.orchestra.run.smithi002.stdout:5882 read 0x5d1a639 thru 0x5d25030 (0xa9f8 bytes) 2024-09-22T01:02:43.962 INFO:teuthology.orchestra.run.smithi002.stdout:5883 writesame 0x7a5925f thru 0x7a60acd data_size 0x2b(0x786f bytes) 2024-09-22T01:02:43.968 INFO:teuthology.orchestra.run.smithi002.stdout:5884 read 0x706fd57 thru 0x707ee95 (0xf13f bytes) 2024-09-22T01:02:43.968 INFO:teuthology.orchestra.run.smithi002.stdout:5885 writesame 0xa0c2635 thru 0xa0c85ed data_size 0x1d(0x5fb9 bytes) 2024-09-22T01:02:43.975 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.975 INFO:teuthology.orchestra.run.smithi002.stdout:5887 read 0x99a6f0e thru 0x99b1bb1 (0xaca4 bytes) 2024-09-22T01:02:43.975 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:43.975 INFO:teuthology.orchestra.run.smithi002.stdout:5889 punch from 0xc8f6398 to 0xc8ff4b0, (0x9118 bytes) 2024-09-22T01:02:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:5890 trunc from 0xe5a5228 to 0x24bd13b 2024-09-22T01:02:44.035 INFO:teuthology.orchestra.run.smithi002.stdout:5892 trunc from 0x24bd13b to 0xb33310a 2024-09-22T01:02:44.041 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:44.037+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:44.042 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:44.042 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:44.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:44.043 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:44.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.043 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3.262a7b13 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:44.043 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:44.043 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 40.269235975000001, 2024-09-22T01:02:44.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366911223, 2024-09-22T01:02:44.044 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:44.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.045 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:44.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.045 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:44.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:44.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771094+0000", 2024-09-22T01:02:44.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0930000000000001e-06 2024-09-22T01:02:44.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771103+0000", 2024-09-22T01:02:44.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0629999999999995e-06 2024-09-22T01:02:44.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771127+0000", 2024-09-22T01:02:44.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3924e-05 2024-09-22T01:02:44.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771147+0000", 2024-09-22T01:02:44.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:02:44.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771214+0000", 2024-09-22T01:02:44.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7348999999999995e-05 2024-09-22T01:02:44.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:44.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771220+0000", 2024-09-22T01:02:44.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3709999999999996e-06 2024-09-22T01:02:44.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.137980+0000", 2024-09-22T01:02:44.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366760011 2024-09-22T01:02:44.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138003+0000", 2024-09-22T01:02:44.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2823000000000002e-05 2024-09-22T01:02:44.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.054 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.054 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3.c7ddc9b3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:44.055 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:44.055 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 40.269148229999999, 2024-09-22T01:02:44.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.367176741, 2024-09-22T01:02:44.055 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:44.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:44.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:44.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:44.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771181+0000", 2024-09-22T01:02:44.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:02:44.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771231+0000", 2024-09-22T01:02:44.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-22T01:02:44.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771247+0000", 2024-09-22T01:02:44.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5858999999999998e-05 2024-09-22T01:02:44.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771259+0000", 2024-09-22T01:02:44.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2814e-05 2024-09-22T01:02:44.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771320+0000", 2024-09-22T01:02:44.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0393999999999998e-05 2024-09-22T01:02:44.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:44.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771325+0000", 2024-09-22T01:02:44.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-22T01:02:44.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138340+0000", 2024-09-22T01:02:44.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3670150409999999 2024-09-22T01:02:44.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138356+0000", 2024-09-22T01:02:44.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6782999999999998e-05 2024-09-22T01:02:44.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.066 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3:c8de5464:::rbd_data.117c2270b223.000000000000001e:head [sparse-read 0~4194304] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:44.066 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:44.066 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 40.268226532, 2024-09-22T01:02:44.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366676402, 2024-09-22T01:02:44.067 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.067 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.067 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:44.067 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.067 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:44.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.068 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:44.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:44.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772103+0000", 2024-09-22T01:02:44.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9929999999999998e-06 2024-09-22T01:02:44.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772113+0000", 2024-09-22T01:02:44.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.499e-06 2024-09-22T01:02:44.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772138+0000", 2024-09-22T01:02:44.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4743999999999999e-05 2024-09-22T01:02:44.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772151+0000", 2024-09-22T01:02:44.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3189e-05 2024-09-22T01:02:44.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772415+0000", 2024-09-22T01:02:44.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026382499999999998 2024-09-22T01:02:44.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:44.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772427+0000", 2024-09-22T01:02:44.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2103000000000001e-05 2024-09-22T01:02:44.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138485+0000", 2024-09-22T01:02:44.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366057984 2024-09-22T01:02:44.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138778+0000", 2024-09-22T01:02:44.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029306500000000001 2024-09-22T01:02:44.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.077 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.077 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3:cd93bbe3:::rbd_data.117c2270b223.0000000000000038:head [sparse-read 0~3145728] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:44.077 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:44.078 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 40.268150417000001, 2024-09-22T01:02:44.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366907898, 2024-09-22T01:02:44.078 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:44.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:44.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.079 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:44.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:44.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:44.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5600000000000005e-07 2024-09-22T01:02:44.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772183+0000", 2024-09-22T01:02:44.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:02:44.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772195+0000", 2024-09-22T01:02:44.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1533e-05 2024-09-22T01:02:44.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772206+0000", 2024-09-22T01:02:44.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1647e-05 2024-09-22T01:02:44.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772493+0000", 2024-09-22T01:02:44.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028702699999999997 2024-09-22T01:02:44.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:44.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772511+0000", 2024-09-22T01:02:44.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7609999999999999e-05 2024-09-22T01:02:44.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138879+0000", 2024-09-22T01:02:44.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3663676759999999 2024-09-22T01:02:44.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.139085+0000", 2024-09-22T01:02:44.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020690899999999999 2024-09-22T01:02:44.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.088 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.089 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:90676 3.9 3:963e58c3:::rbd_data.117c77dccaa7.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-22T01:02:44.089 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:44.089 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.498218764999997, 2024-09-22T01:02:44.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093363432999999996, 2024-09-22T01:02:44.089 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:44.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 90676 2024-09-22T01:02:44.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.091 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:44.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:44.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542112+0000", 2024-09-22T01:02:44.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9730000000000002e-06 2024-09-22T01:02:44.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542126+0000", 2024-09-22T01:02:44.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:02:44.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542161+0000", 2024-09-22T01:02:44.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4975000000000003e-05 2024-09-22T01:02:44.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542185+0000", 2024-09-22T01:02:44.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4613999999999998e-05 2024-09-22T01:02:44.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542321+0000", 2024-09-22T01:02:44.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013617999999999999 2024-09-22T01:02:44.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542562+0000", 2024-09-22T01:02:44.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024014899999999999 2024-09-22T01:02:44.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:44.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.556142+0000", 2024-09-22T01:02:44.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013580126 2024-09-22T01:02:44.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:44.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635270+0000", 2024-09-22T01:02:44.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079128575000000007 2024-09-22T01:02:44.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:44.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635299+0000", 2024-09-22T01:02:44.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8996000000000002e-05 2024-09-22T01:02:44.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:44.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635391+0000", 2024-09-22T01:02:44.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1712000000000006e-05 2024-09-22T01:02:44.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635473+0000", 2024-09-22T01:02:44.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1588999999999994e-05 2024-09-22T01:02:44.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.102 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.102 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104388 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [write 9699485~40067] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:02:44.102 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 21.10759672, 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896004440000002, 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104388 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932733+0000", 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1780000000000002e-06 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932760+0000", 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6174e-05 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932775+0000", 2024-09-22T01:02:44.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5603e-05 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932787+0000", 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2155e-05 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932873+0000", 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5391999999999998e-05 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932887+0000", 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3915e-05 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930412+0000", 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.99752547400000002 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930446+0000", 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3841999999999999e-05 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.019619+0000", 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.089172569 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020035+0000", 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041613899999999999 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020214+0000", 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017929999999999999 2024-09-22T01:02:44.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.021438+0000", 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012240440000000001 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022156+0000", 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071828699999999996 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022231+0000", 2024-09-22T01:02:44.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420999999999998e-05 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022332+0000", 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100951 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106194 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 778240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:44.110 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.656754124000001, 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091629434999999995, 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106194 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383577+0000", 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7549999999999999e-06 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383585+0000", 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4209999999999998e-06 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383599+0000", 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4599e-05 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.474958+0000", 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091358710999999995 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475039+0000", 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1075999999999994e-05 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475203+0000", 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016427600000000001 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106195 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.656719281000001, 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091906246999999996, 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106195 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:44.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383612+0000", 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383620+0000", 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0600000000000008e-06 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383626+0000", 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8529999999999997e-06 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475245+0000", 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091618623999999996 2024-09-22T01:02:44.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475326+0000", 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1166000000000002e-05 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475515+0000", 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018890399999999999 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106196 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.656693258000001, 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092159059000000002, 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106196 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383645+0000", 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7119999999999995e-06 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383653+0000", 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.807e-06 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383658+0000", 2024-09-22T01:02:44.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.04e-06 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475547+0000", 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091889557999999996 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475630+0000", 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3114000000000004e-05 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475794+0000", 2024-09-22T01:02:44.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016325199999999999 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106197 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 821248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.656660558, 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092422578000000005, 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106197 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:44.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383668+0000", 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.63e-07 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383677+0000", 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8859999999999993e-06 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383685+0000", 2024-09-22T01:02:44.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5609999999999999e-06 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383690+0000", 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475825+0000", 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092134439999999998 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475917+0000", 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2559000000000006e-05 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476090+0000", 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017251899999999999 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106198 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:44.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.656627396999999, 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092626416000000003, 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106198 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:44.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384316+0000", 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061580100000000002 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384323+0000", 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384338+0000", 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5126999999999999e-05 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384348+0000", 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8479999999999996e-06 2024-09-22T01:02:44.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476109+0000", 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091760280999999999 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476178+0000", 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9865999999999998e-05 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476327+0000", 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014848199999999999 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106199 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.65596298, 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092274072999999998, 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106199 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385301+0000", 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093565 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385312+0000", 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1589e-05 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385342+0000", 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9743999999999998e-05 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385361+0000", 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9219999999999999e-05 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476358+0000", 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090996442999999996 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476438+0000", 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9977999999999999e-05 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476639+0000", 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000201449 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106200 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.654934992999999, 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091589137000000001, 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106200 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385394+0000", 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1599999999999999e-06 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385401+0000", 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9090000000000003e-06 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385420+0000", 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:02:44.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385434+0000", 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4068e-05 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476677+0000", 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091242904999999999 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476760+0000", 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3783999999999996e-05 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476982+0000", 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022163200000000001 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106201 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.654713508, 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091715669, 2024-09-22T01:02:44.141 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106201 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385616+0000", 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.277e-06 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385624+0000", 2024-09-22T01:02:44.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7910000000000005e-06 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385646+0000", 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2251000000000001e-05 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385680+0000", 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3933000000000002e-05 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477014+0000", 2024-09-22T01:02:44.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091334175000000004 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477096+0000", 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2361999999999995e-05 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477330+0000", 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023388 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106202 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.654625091, 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091937313000000007, 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106202 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385835+0000", 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013252699999999999 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385844+0000", 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1599999999999998e-06 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385864+0000", 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0579000000000001e-05 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385879+0000", 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4871e-05 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477362+0000", 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091483156999999996 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477452+0000", 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9868000000000001e-05 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477640+0000", 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018815100000000001 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106203 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.654406258, 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092020950000000004, 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106203 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386095+0000", 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017321200000000001 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386104+0000", 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5109999999999992e-06 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386129+0000", 2024-09-22T01:02:44.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4785000000000001e-05 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386145+0000", 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6191000000000001e-05 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477675+0000", 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091529570000000005 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477762+0000", 2024-09-22T01:02:44.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6861999999999996e-05 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477943+0000", 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180819 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106204 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.65415582, 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092064394999999993, 2024-09-22T01:02:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106204 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386406+0000", 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023343699999999999 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386412+0000", 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386425+0000", 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2719e-05 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386433+0000", 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2810000000000008e-06 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477976+0000", 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091542739999999997 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478069+0000", 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2971000000000001e-05 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478237+0000", 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167949 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106205 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.158 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.653880152999999, 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092018127000000005, 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106205 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387017+0000", 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056957000000000004 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387024+0000", 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387035+0000", 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1187e-05 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387045+0000", 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0750000000000004e-06 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478259+0000", 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091214394000000004 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478311+0000", 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1984000000000002e-05 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478466+0000", 2024-09-22T01:02:44.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015504200000000001 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106206 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.653264767, 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091679640000000007, 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106206 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387533+0000", 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047020100000000001 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387539+0000", 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8200000000000002e-06 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387552+0000", 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2865e-05 2024-09-22T01:02:44.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387560+0000", 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5499999999999997e-06 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478507+0000", 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090947741999999998 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478599+0000", 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1181000000000007e-05 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478743+0000", 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014428100000000001 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106208 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:44.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.652715964, 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091664238999999995, 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106208 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:44.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391033+0000", 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00342144 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391040+0000", 2024-09-22T01:02:44.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391054+0000", 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4089e-05 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391063+0000", 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3809999999999999e-06 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479014+0000", 2024-09-22T01:02:44.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087951647999999993 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479099+0000", 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4859999999999997e-05 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479276+0000", 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017700699999999999 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.171 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:44.173 INFO:teuthology.orchestra.run.smithi002.stdout:5893 read 0x4974724 thru 0x4978dbc (0x4699 bytes) 2024-09-22T01:02:44.173 INFO:teuthology.orchestra.run.smithi002.stdout:5894 writesame 0x8abfac8 thru 0x8ac59f1 data_size 0xd(0x5f2a bytes) 2024-09-22T01:02:44.173 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:44.173 INFO:teuthology.orchestra.run.smithi002.stdout:5896 read 0xa7dc60f thru 0xa7ea354 (0xdd46 bytes) 2024-09-22T01:02:44.173 INFO:teuthology.orchestra.run.smithi002.stdout:5897 punch from 0x4d05022 to 0x4d0fb91, (0xab6f bytes) 2024-09-22T01:02:44.173 INFO:teuthology.orchestra.run.smithi002.stdout:5898 write 0x42fe9f8 thru 0x430dab0 (0xf0b9 bytes) 2024-09-22T01:02:44.173 INFO:teuthology.orchestra.run.smithi002.stdout:5900 write 0x6d60d3 thru 0x6defd3 (0x8f01 bytes) 2024-09-22T01:02:44.174 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:44.176 INFO:teuthology.orchestra.run.smithi002.stdout:5903 punch from 0x710ee60 to 0x711721b, (0x83bb bytes) 2024-09-22T01:02:44.176 INFO:teuthology.orchestra.run.smithi002.stdout:5904 writesame 0x4ae8c76 thru 0x4aeec47 data_size 0x37(0x5fd2 bytes) 2024-09-22T01:02:44.176 INFO:teuthology.orchestra.run.smithi002.stdout:5906 write 0x7d9a7a3 thru 0x7da5f32 (0xb790 bytes) 2024-09-22T01:02:44.176 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:44.176 INFO:teuthology.orchestra.run.smithi002.stdout:5909 write 0x597098b thru 0x59731dd (0x2853 bytes) 2024-09-22T01:02:44.176 INFO:teuthology.orchestra.run.smithi002.stdout:5910 write 0x74bbacb thru 0x74c293b (0x6e71 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5911 punch from 0x344e41a to 0x3451462, (0x3048 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5915 read 0xa007dff thru 0xa00be25 (0x4027 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5917 punch from 0x554f008 to 0x5550c44, (0x1c3c bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5918 write 0xa17f763 thru 0xa1861b1 (0x6a4f bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5920 punch from 0x2ec9007 to 0x2ece247, (0x5240 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5921 read 0x67efe6a thru 0x67fc7bf (0xc956 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5925 trunc from 0xb33310a to 0x94cb3a0 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5926 writesame 0x96fb40e thru 0x9708b85 data_size 0x46(0xd778 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5927 read 0x4026929 thru 0x40281ca (0x18a2 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5928 write 0x8bde3a1 thru 0x8be2658 (0x42b8 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5930 write 0xaf77703 thru 0xaf86a72 (0xf370 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5932 punch from 0xa999f39 to 0xa9a5c11, (0xbcd8 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5933 write 0x5cd6c82 thru 0x5ce6154 (0xf4d3 bytes) 2024-09-22T01:02:44.177 INFO:teuthology.orchestra.run.smithi002.stdout:5934 write 0xa584892 thru 0xa58f9c4 (0xb133 bytes) 2024-09-22T01:02:44.178 INFO:teuthology.orchestra.run.smithi002.stdout:5935 write 0xd1f4172 thru 0xd20211a (0xdfa9 bytes) 2024-09-22T01:02:44.178 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:44.169+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:44.183 INFO:teuthology.orchestra.run.smithi002.stdout:5936 write 0x4400bb3 thru 0x4408491 (0x78df bytes) 2024-09-22T01:02:44.188 INFO:teuthology.orchestra.run.smithi002.stdout:5937 punch from 0x4bdc3d6 to 0x4bebd54, (0xf97e bytes) 2024-09-22T01:02:44.188 INFO:teuthology.orchestra.run.smithi002.stdout:5938 read 0x6477c3d thru 0x6483ee1 (0xc2a5 bytes) 2024-09-22T01:02:44.197 INFO:teuthology.orchestra.run.smithi002.stdout:5939 write 0xd68ef43 thru 0xd69eee6 (0xffa4 bytes) 2024-09-22T01:02:44.204 INFO:teuthology.orchestra.run.smithi002.stdout:5942 punch from 0x682c852 to 0x682f7d8, (0x2f86 bytes) 2024-09-22T01:02:44.204 INFO:teuthology.orchestra.run.smithi002.stdout:5943 punch from 0xc7b3b94 to 0xc7b8b9c, (0x5008 bytes) 2024-09-22T01:02:44.204 INFO:teuthology.orchestra.run.smithi002.stdout:5944 write 0x6abc8b2 thru 0x6abd93d (0x108c bytes) 2024-09-22T01:02:44.210 INFO:teuthology.orchestra.run.smithi002.stdout:5945 punch from 0x61826d0 to 0x6183878, (0x11a8 bytes) 2024-09-22T01:02:44.210 INFO:teuthology.orchestra.run.smithi002.stdout:5950 read 0x6dc6ec7 thru 0x6dd0717 (0x9851 bytes) 2024-09-22T01:02:44.210 INFO:teuthology.orchestra.run.smithi002.stdout:5951 write 0xc4d581 thru 0xc4f337 (0x1db7 bytes) 2024-09-22T01:02:44.217 INFO:teuthology.orchestra.run.smithi002.stdout:5953 punch from 0x17c88a6 to 0x17cebf3, (0x634d bytes) 2024-09-22T01:02:44.217 INFO:teuthology.orchestra.run.smithi002.stdout:5954 punch from 0xaad8712 to 0xaae5467, (0xcd55 bytes) 2024-09-22T01:02:44.217 INFO:teuthology.orchestra.run.smithi002.stdout:5956 writesame 0x765f772 thru 0x7662126 data_size 0x1(0x29b5 bytes) 2024-09-22T01:02:44.236 INFO:teuthology.orchestra.run.smithi002.stdout:5957 write 0x5d114c6 thru 0x5d1dd7f (0xc8ba bytes) 2024-09-22T01:02:44.243 INFO:teuthology.orchestra.run.smithi002.stdout:5959 read 0x54c4b75 thru 0x54c95d9 (0x4a65 bytes) 2024-09-22T01:02:44.243 INFO:teuthology.orchestra.run.smithi002.stdout:5960 read 0x6b92377 thru 0x6b9f0ed (0xcd77 bytes) 2024-09-22T01:02:44.261 INFO:teuthology.orchestra.run.smithi002.stdout:5961 trunc from 0xd69eee7 to 0x312e305 2024-09-22T01:02:44.273 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:44.269+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:44.306 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:44.306 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:44.306 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:44.306 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:5962 writesame 0xbbd7b65 thru 0xbbe6800 data_size 0x26(0xec9c bytes) 2024-09-22T01:02:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:5963 write 0xb475b5b thru 0xb47c73a (0x6be0 bytes) 2024-09-22T01:02:44.318 INFO:teuthology.orchestra.run.smithi002.stdout:5964 read 0x19f993d thru 0x1a05939 (0xbffd bytes) 2024-09-22T01:02:44.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:44.331 INFO:teuthology.orchestra.run.smithi002.stdout:5966 write 0xdea24ea thru 0xdeaca24 (0xa53b bytes) 2024-09-22T01:02:44.337 INFO:teuthology.orchestra.run.smithi002.stdout:5967 write 0xb5c4935 thru 0xb5d0381 (0xba4d bytes) 2024-09-22T01:02:44.344 INFO:teuthology.orchestra.run.smithi002.stdout:5968 read 0x4c08672 thru 0x4c12c85 (0xa614 bytes) 2024-09-22T01:02:44.344 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:44.344 INFO:teuthology.orchestra.run.smithi002.stdout:5970 read 0x10ab849 thru 0x10b47f1 (0x8fa9 bytes) 2024-09-22T01:02:44.344 INFO:teuthology.orchestra.run.smithi002.stdout:5972 write 0x9454cf thru 0x94fd3f (0xa871 bytes) 2024-09-22T01:02:44.351 INFO:teuthology.orchestra.run.smithi002.stdout:5973 read 0xbeda99c thru 0xbedaa12 (0x77 bytes) 2024-09-22T01:02:44.351 INFO:teuthology.orchestra.run.smithi002.stdout:5975 writesame 0x222ba12 thru 0x2239169 data_size 0xc(0xd758 bytes) 2024-09-22T01:02:44.362 INFO:teuthology.orchestra.run.smithi002.stdout:5976 punch from 0xb2f9ee6 to 0xb301914, (0x7a2e bytes) 2024-09-22T01:02:44.363 INFO:teuthology.orchestra.run.smithi002.stdout:5978 writesame 0x3f243f1 thru 0x3f2b489 data_size 0x5(0x7099 bytes) 2024-09-22T01:02:44.371 INFO:teuthology.orchestra.run.smithi002.stdout:5979 write 0x2f82df7 thru 0x2f8c39e (0x95a8 bytes) 2024-09-22T01:02:44.374 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:44.369+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:44.377 INFO:teuthology.orchestra.run.smithi002.stdout:5980 read 0x8cd5d72 thru 0x8ce5a40 (0xfccf bytes) 2024-09-22T01:02:44.377 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:44.377 INFO:teuthology.orchestra.run.smithi002.stdout:5982 writesame 0x9bdaed7 thru 0x9be7861 data_size 0x11(0xc98b bytes) 2024-09-22T01:02:44.386 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-22T01:02:44.386 INFO:teuthology.orchestra.run.smithi002.stdout:5984 read 0x4d8b08b thru 0x4d95a1f (0xa995 bytes) 2024-09-22T01:02:44.387 INFO:teuthology.orchestra.run.smithi002.stdout:5987 flatten 2024-09-22T01:02:44.407 INFO:teuthology.orchestra.run.smithi002.stdout:5988 write 0x1a65e2f thru 0x1a66bb4 (0xd86 bytes) 2024-09-22T01:02:44.427 INFO:teuthology.orchestra.run.smithi002.stdout:5989 trunc from 0xdeaca25 to 0x4e68361 2024-09-22T01:02:44.449 INFO:teuthology.orchestra.run.smithi002.stdout:5990 punch from 0x1dc5275 to 0x1dc6422, (0x11ad bytes) 2024-09-22T01:02:44.449 INFO:teuthology.orchestra.run.smithi002.stdout:5991 punch from 0x24c98c6 to 0x24d5c4d, (0xc387 bytes) 2024-09-22T01:02:44.474 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:44.473+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:44.475 INFO:teuthology.orchestra.run.smithi002.stdout:5992 write 0xc3c55df thru 0xc3c79c5 (0x23e7 bytes) 2024-09-22T01:02:44.482 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:44.482 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:44.482 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:44.482 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:44.482 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:44.484 INFO:teuthology.orchestra.run.smithi002.stdout:5993 trunc from 0xc3c79c6 to 0xd46de7a 2024-09-22T01:02:44.494 INFO:teuthology.orchestra.run.smithi002.stdout:5994 write 0xa5f22d5 thru 0xa5f48b6 (0x25e2 bytes) 2024-09-22T01:02:44.500 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:44.501 INFO:teuthology.orchestra.run.smithi002.stdout:5996 punch from 0xac0a337 to 0xac0c882, (0x254b bytes) 2024-09-22T01:02:44.501 INFO:teuthology.orchestra.run.smithi002.stdout:5997 writesame 0x451d790 thru 0x4528133 data_size 0x54(0xa9a4 bytes) 2024-09-22T01:02:44.509 INFO:teuthology.orchestra.run.smithi002.stdout:6000 write 0x5d3da6 thru 0x5e02ee (0xc549 bytes) 2024-09-22T01:02:44.542 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-22T01:02:44.575 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:44.569+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141633710000001, 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:44.617 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:44.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:44.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141612881, 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:44.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:44.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141584365999996, 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:44.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:44.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.630 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141561715999998, 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:44.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141539721000001, 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:44.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:44.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141514657000002, 2024-09-22T01:02:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.643 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141492223999997, 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:44.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:44.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141470747, 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.141441989, 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.140628997, 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:44.656 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:44.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:44.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.140604312000001, 2024-09-22T01:02:44.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136282778000002, 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:44.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.669 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136253926999998, 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136232202000002, 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.678 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136211772000003, 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136189455999997, 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.683 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:44.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:44.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:44.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136121660999997, 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:44.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136088524000002, 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:44.691 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136064601999998, 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:44.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 63.136041005000003, 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:44.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:44.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:44.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:44.704 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:44.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:44.706 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:44.701+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:44.735 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:44.785 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.7 to 1 (10000) 2024-09-22T01:02:44.807 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T01:02:44.807 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:44.805+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:44.836 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:44.837 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:44.837 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:44.837 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:44.851 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:44.894 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:44.894 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:44.908 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:44.905+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:44.913 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:44.913 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:44.931 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:44.931 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:44.948 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:44.948 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:44.972 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:44.973 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:44.976 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:44.977 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:44.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:45.006 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:45.006 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:45.037 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:45.033+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:45.039 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:45.039 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:45.060 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:45.060 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:45.118 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:45.117+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:45.121 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:45.121 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:45.135 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:45.225 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:45.221+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:45.298 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:45.298 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:45.312 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:45.353 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:45.349+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:45.419 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:45.413+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:45.478 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:45.479 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:45.492 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:45.538 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:45.533+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:45.625 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:45.621+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:45.653 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:45.654 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:45.668 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:45.726 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:45.725+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:45.789 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:45.789 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:45.801 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T01:02:45.803 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:02:45.805 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:45.829 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T01:02:45.829 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:02:45.830 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:45.825+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:45.927 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:45.925+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:45.940 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:45.940 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:45.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:46.030 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:46.029+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:46.078 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:46.078 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:46.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:46.137 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:46.133+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:46.181 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:02:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-22T01:02:45.794545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1540855080,"num_objects":1926,"num_object_clones":586,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":744,"num_read":190915,"num_read_kb":747090,"num_write":20486,"num_write_kb":411627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":175106042,"num_keys_recovered":760,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16391,"ondisk_log_size":16391,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":870512,"kb_used_data":528820,"kb_used_omap":13,"kb_used_meta":341362,"kb_avail":659732368,"statfs":{"total":676457349120,"available":675565944832,"internally_reserved":0,"allocated":541511680,"data_stored":635565445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":349554932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-885071,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3945,"num_read_kb":7718,"num_write":1035,"num_write_kb":5655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031702"},"pg_stats":[{"pgid":"3.1f","version":"118'440","reported_seq":6708,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961841+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:02:44.961841+0000","last_peered":"2024-09-22T01:02:44.961841+0000","last_clean":"2024-09-22T01:02:44.961841+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:02:44.961841+0000","last_undegraded":"2024-09-22T01:02:44.961841+0000","last_fullsized":"2024-09-22T01:02:44.961841+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:38.421694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":72398686,"num_objects":68,"num_object_clones":21,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":19,"num_read":6471,"num_read_kb":45914,"num_write":501,"num_write_kb":17789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"119'640","reported_seq":2950,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.083658+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:02:45.083658+0000","last_peered":"2024-09-22T01:02:45.083658+0000","last_clean":"2024-09-22T01:02:45.083658+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:02:45.083658+0000","last_undegraded":"2024-09-22T01:02:45.083658+0000","last_fullsized":"2024-09-22T01:02:45.083658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:11:31.811392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":28882882,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":5717,"num_read_kb":12441,"num_write":886,"num_write_kb":7177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"115'403","reported_seq":4482,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.959807+0000","last_change":"2024-09-22T01:01:43.138890+0000","last_active":"2024-09-22T01:02:40.959807+0000","last_peered":"2024-09-22T01:02:40.959807+0000","last_clean":"2024-09-22T01:02:40.959807+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:02:40.959807+0000","last_undegraded":"2024-09-22T01:02:40.959807+0000","last_fullsized":"2024-09-22T01:02:40.959807+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:57.315255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83307193,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":28,"num_read":4780,"num_read_kb":31757,"num_write":448,"num_write_kb":17874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"115'956","reported_seq":9662,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.851291+0000","last_change":"2024-09-22T01:02:39.740146+0000","last_active":"2024-09-22T01:02:40.851291+0000","last_peered":"2024-09-22T01:02:40.851291+0000","last_clean":"2024-09-22T01:02:40.851291+0000","last_became_active":"2024-09-22T01:02:39.738124+0000","last_became_peered":"2024-09-22T01:02:39.738124+0000","last_unstale":"2024-09-22T01:02:40.851291+0000","last_undegraded":"2024-09-22T01:02:40.851291+0000","last_fullsized":"2024-09-22T01:02:40.851291+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":956,"log_dups_size":0,"ondisk_log_size":956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:26:18.864536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84842087,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":23,"num_read":11110,"num_read_kb":51376,"num_write":1445,"num_write_kb":18203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":31958297,"num_keys_recovered":274,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"119'987","reported_seq":2321,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.081038+0000","last_change":"2024-09-22T01:01:43.147132+0000","last_active":"2024-09-22T01:02:45.081038+0000","last_peered":"2024-09-22T01:02:45.081038+0000","last_clean":"2024-09-22T01:02:45.081038+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:02:45.081038+0000","last_undegraded":"2024-09-22T01:02:45.081038+0000","last_fullsized":"2024-09-22T01:02:45.081038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":987,"log_dups_size":0,"ondisk_log_size":987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:36:47.660668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":32243249,"num_objects":63,"num_object_clones":24,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":4965,"num_read_kb":15217,"num_write":1461,"num_write_kb":8369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"119'402","reported_seq":11504,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.065778+0000","last_change":"2024-09-22T01:02:39.739952+0000","last_active":"2024-09-22T01:02:45.065778+0000","last_peered":"2024-09-22T01:02:45.065778+0000","last_clean":"2024-09-22T01:02:45.065778+0000","last_became_active":"2024-09-22T01:02:39.738141+0000","last_became_peered":"2024-09-22T01:02:39.738141+0000","last_unstale":"2024-09-22T01:02:45.065778+0000","last_undegraded":"2024-09-22T01:02:45.065778+0000","last_fullsized":"2024-09-22T01:02:45.065778+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:08.091112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":63093539,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":11619,"num_read_kb":25493,"num_write":502,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":7836182,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"118'343","reported_seq":9624,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784426+0000","last_change":"2024-09-22T01:02:37.738989+0000","last_active":"2024-09-22T01:02:44.784426+0000","last_peered":"2024-09-22T01:02:44.784426+0000","last_clean":"2024-09-22T01:02:44.784426+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:02:44.784426+0000","last_undegraded":"2024-09-22T01:02:44.784426+0000","last_fullsized":"2024-09-22T01:02:44.784426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:52:16.421270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":38522760,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":9088,"num_read_kb":31273,"num_write":422,"num_write_kb":9303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"119'635","reported_seq":5845,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990180+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:02:44.990180+0000","last_peered":"2024-09-22T01:02:44.990180+0000","last_clean":"2024-09-22T01:02:44.990180+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:02:44.990180+0000","last_undegraded":"2024-09-22T01:02:44.990180+0000","last_fullsized":"2024-09-22T01:02:44.990180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:01:35.649602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":84264818,"num_objects":71,"num_object_clones":22,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":18,"num_read":7976,"num_read_kb":42180,"num_write":753,"num_write_kb":16828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"119'452","reported_seq":3751,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961897+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:02:44.961897+0000","last_peered":"2024-09-22T01:02:44.961897+0000","last_clean":"2024-09-22T01:02:44.961897+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:02:44.961897+0000","last_undegraded":"2024-09-22T01:02:44.961897+0000","last_fullsized":"2024-09-22T01:02:44.961897+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:32:30.790477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":39597315,"num_objects":63,"num_object_clones":23,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":4515,"num_read_kb":18057,"num_write":572,"num_write_kb":8510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"114'298","reported_seq":1794,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.796009+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:02:40.796009+0000","last_peered":"2024-09-22T01:02:40.796009+0000","last_clean":"2024-09-22T01:02:40.796009+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:02:40.796009+0000","last_undegraded":"2024-09-22T01:02:40.796009+0000","last_fullsized":"2024-09-22T01:02:40.796009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:17.208328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81626369,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":20,"num_read":2564,"num_read_kb":34277,"num_write":357,"num_write_kb":28683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"118'545","reported_seq":6081,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960197+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:02:44.960197+0000","last_peered":"2024-09-22T01:02:44.960197+0000","last_clean":"2024-09-22T01:02:44.960197+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:02:44.960197+0000","last_undegraded":"2024-09-22T01:02:44.960197+0000","last_fullsized":"2024-09-22T01:02:44.960197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:36:59.661355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":33697818,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":8167,"num_read_kb":26512,"num_write":599,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"118'455","reported_seq":13392,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960315+0000","last_change":"2024-09-22T01:01:43.139393+0000","last_active":"2024-09-22T01:02:44.960315+0000","last_peered":"2024-09-22T01:02:44.960315+0000","last_clean":"2024-09-22T01:02:44.960315+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:02:44.960315+0000","last_undegraded":"2024-09-22T01:02:44.960315+0000","last_fullsized":"2024-09-22T01:02:44.960315+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:43:32.338655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":68935816,"num_objects":65,"num_object_clones":24,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":13638,"num_read_kb":40761,"num_write":509,"num_write_kb":17919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":207,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.761521+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:02:40.761521+0000","last_peered":"2024-09-22T01:02:40.761521+0000","last_clean":"2024-09-22T01:02:40.761521+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:02:40.761521+0000","last_undegraded":"2024-09-22T01:02:40.761521+0000","last_fullsized":"2024-09-22T01:02:40.761521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"119'435","reported_seq":2488,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961736+0000","last_change":"2024-09-22T01:02:37.739819+0000","last_active":"2024-09-22T01:02:44.961736+0000","last_peered":"2024-09-22T01:02:44.961736+0000","last_clean":"2024-09-22T01:02:44.961736+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:02:44.961736+0000","last_undegraded":"2024-09-22T01:02:44.961736+0000","last_fullsized":"2024-09-22T01:02:44.961736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:30:12.181033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":21061190,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":3612,"num_read_kb":6109,"num_write":468,"num_write_kb":3304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784723+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:02:44.784723+0000","last_peered":"2024-09-22T01:02:44.784723+0000","last_clean":"2024-09-22T01:02:44.784723+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:02:44.784723+0000","last_undegraded":"2024-09-22T01:02:44.784723+0000","last_fullsized":"2024-09-22T01:02:44.784723+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"118'497","reported_seq":3088,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990292+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:02:44.990292+0000","last_peered":"2024-09-22T01:02:44.990292+0000","last_clean":"2024-09-22T01:02:44.990292+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:02:44.990292+0000","last_undegraded":"2024-09-22T01:02:44.990292+0000","last_fullsized":"2024-09-22T01:02:44.990292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:48:57.425985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":35266930,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":4214,"num_read_kb":12482,"num_write":649,"num_write_kb":6967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960834+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:02:44.960834+0000","last_peered":"2024-09-22T01:02:44.960834+0000","last_clean":"2024-09-22T01:02:44.960834+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:02:44.960834+0000","last_undegraded":"2024-09-22T01:02:44.960834+0000","last_fullsized":"2024-09-22T01:02:44.960834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"118'355","reported_seq":3947,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784621+0000","last_change":"2024-09-22T01:02:38.738598+0000","last_active":"2024-09-22T01:02:44.784621+0000","last_peered":"2024-09-22T01:02:44.784621+0000","last_clean":"2024-09-22T01:02:44.784621+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:02:44.784621+0000","last_undegraded":"2024-09-22T01:02:44.784621+0000","last_fullsized":"2024-09-22T01:02:44.784621+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:47.050434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":40666215,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":30,"num_read":3788,"num_read_kb":26299,"num_write":376,"num_write_kb":16759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784507+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:02:44.784507+0000","last_peered":"2024-09-22T01:02:44.784507+0000","last_clean":"2024-09-22T01:02:44.784507+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:02:44.784507+0000","last_undegraded":"2024-09-22T01:02:44.784507+0000","last_fullsized":"2024-09-22T01:02:44.784507+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"115'502","reported_seq":3661,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.795917+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:02:40.795917+0000","last_peered":"2024-09-22T01:02:40.795917+0000","last_clean":"2024-09-22T01:02:40.795917+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:02:40.795917+0000","last_undegraded":"2024-09-22T01:02:40.795917+0000","last_fullsized":"2024-09-22T01:02:40.795917+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:50.990905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54122868,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":18,"num_read":6107,"num_read_kb":11468,"num_write":466,"num_write_kb":5346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"119'459","reported_seq":2085,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.057741+0000","last_change":"2024-09-22T01:01:43.148315+0000","last_active":"2024-09-22T01:02:45.057741+0000","last_peered":"2024-09-22T01:02:45.057741+0000","last_clean":"2024-09-22T01:02:45.057741+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:02:45.057741+0000","last_undegraded":"2024-09-22T01:02:45.057741+0000","last_fullsized":"2024-09-22T01:02:45.057741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:33:05.221914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":54606605,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":2891,"num_read_kb":17537,"num_write":514,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":180,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.795668+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:02:40.795668+0000","last_peered":"2024-09-22T01:02:40.795668+0000","last_clean":"2024-09-22T01:02:40.795668+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:02:40.795668+0000","last_undegraded":"2024-09-22T01:02:40.795668+0000","last_fullsized":"2024-09-22T01:02:40.795668+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"119'1546","reported_seq":3364,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784518+0000","last_change":"2024-09-22T01:02:39.740320+0000","last_active":"2024-09-22T01:02:44.784518+0000","last_peered":"2024-09-22T01:02:44.784518+0000","last_clean":"2024-09-22T01:02:44.784518+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:02:44.784518+0000","last_undegraded":"2024-09-22T01:02:44.784518+0000","last_fullsized":"2024-09-22T01:02:44.784518+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1546,"log_dups_size":0,"ondisk_log_size":1546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:56.118200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":49656345,"num_objects":67,"num_object_clones":23,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":8866,"num_read_kb":25283,"num_write":2641,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784066+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:02:44.784066+0000","last_peered":"2024-09-22T01:02:44.784066+0000","last_clean":"2024-09-22T01:02:44.784066+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:02:44.784066+0000","last_undegraded":"2024-09-22T01:02:44.784066+0000","last_fullsized":"2024-09-22T01:02:44.784066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'339","reported_seq":2504,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961699+0000","last_change":"2024-09-22T01:01:43.152419+0000","last_active":"2024-09-22T01:02:44.961699+0000","last_peered":"2024-09-22T01:02:44.961699+0000","last_clean":"2024-09-22T01:02:44.961699+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:02:44.961699+0000","last_undegraded":"2024-09-22T01:02:44.961699+0000","last_fullsized":"2024-09-22T01:02:44.961699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:57.141863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":44140547,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":2187,"num_read_kb":18579,"num_write":371,"num_write_kb":15217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":226,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960134+0000","last_change":"2024-09-22T01:02:35.049954+0000","last_active":"2024-09-22T01:02:44.960134+0000","last_peered":"2024-09-22T01:02:44.960134+0000","last_clean":"2024-09-22T01:02:44.960134+0000","last_became_active":"2024-09-22T01:02:35.049024+0000","last_became_peered":"2024-09-22T01:02:35.049024+0000","last_unstale":"2024-09-22T01:02:44.960134+0000","last_undegraded":"2024-09-22T01:02:44.960134+0000","last_fullsized":"2024-09-22T01:02:44.960134+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:45.992332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"118'328","reported_seq":1932,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990212+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:02:44.990212+0000","last_peered":"2024-09-22T01:02:44.990212+0000","last_clean":"2024-09-22T01:02:44.990212+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:02:44.990212+0000","last_undegraded":"2024-09-22T01:02:44.990212+0000","last_fullsized":"2024-09-22T01:02:44.990212+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:55.260294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":36976643,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":19,"num_read":1621,"num_read_kb":15577,"num_write":344,"num_write_kb":17862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.795673+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:02:40.795673+0000","last_peered":"2024-09-22T01:02:40.795673+0000","last_clean":"2024-09-22T01:02:40.795673+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:02:40.795673+0000","last_undegraded":"2024-09-22T01:02:40.795673+0000","last_fullsized":"2024-09-22T01:02:40.795673+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":367,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.959934+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:02:44.959934+0000","last_peered":"2024-09-22T01:02:44.959934+0000","last_clean":"2024-09-22T01:02:44.959934+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:02:44.959934+0000","last_undegraded":"2024-09-22T01:02:44.959934+0000","last_fullsized":"2024-09-22T01:02:44.959934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"119'360","reported_seq":10029,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990192+0000","last_change":"2024-09-22T01:02:39.736794+0000","last_active":"2024-09-22T01:02:44.990192+0000","last_peered":"2024-09-22T01:02:44.990192+0000","last_clean":"2024-09-22T01:02:44.990192+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:02:44.990192+0000","last_undegraded":"2024-09-22T01:02:44.990192+0000","last_fullsized":"2024-09-22T01:02:44.990192+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:53:34.004734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":60097571,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":28,"num_read":10369,"num_read_kb":27110,"num_write":380,"num_write_kb":13691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"119'491","reported_seq":1921,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.038837+0000","last_change":"2024-09-22T01:01:43.142999+0000","last_active":"2024-09-22T01:02:45.038837+0000","last_peered":"2024-09-22T01:02:45.038837+0000","last_clean":"2024-09-22T01:02:45.038837+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:02:45.038837+0000","last_undegraded":"2024-09-22T01:02:45.038837+0000","last_fullsized":"2024-09-22T01:02:45.038837+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:16:43.137041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":41881935,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":21,"num_read":2593,"num_read_kb":6970,"num_write":588,"num_write_kb":8014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"115'613","reported_seq":15132,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.761511+0000","last_change":"2024-09-22T01:02:37.738638+0000","last_active":"2024-09-22T01:02:40.761511+0000","last_peered":"2024-09-22T01:02:40.761511+0000","last_clean":"2024-09-22T01:02:40.761511+0000","last_became_active":"2024-09-22T01:02:37.737218+0000","last_became_peered":"2024-09-22T01:02:37.737218+0000","last_unstale":"2024-09-22T01:02:40.761511+0000","last_undegraded":"2024-09-22T01:02:40.761511+0000","last_fullsized":"2024-09-22T01:02:40.761511+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:06:56.183242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":62538447,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":21,"num_read":15926,"num_read_kb":39962,"num_write":677,"num_write_kb":12813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":13401696,"num_keys_recovered":167,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"119'565","reported_seq":2256,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.039730+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:02:45.039730+0000","last_peered":"2024-09-22T01:02:45.039730+0000","last_clean":"2024-09-22T01:02:45.039730+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:02:45.039730+0000","last_undegraded":"2024-09-22T01:02:45.039730+0000","last_fullsized":"2024-09-22T01:02:45.039730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:11.769709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":22105578,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":3955,"num_read_kb":19562,"num_write":725,"num_write_kb":10845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"115'336","reported_seq":1001,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.795994+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:02:40.795994+0000","last_peered":"2024-09-22T01:02:40.795994+0000","last_clean":"2024-09-22T01:02:40.795994+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:02:40.795994+0000","last_undegraded":"2024-09-22T01:02:40.795994+0000","last_fullsized":"2024-09-22T01:02:40.795994+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:24.542838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48677873,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":2552,"num_read_kb":21017,"num_write":358,"num_write_kb":10863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"119'633","reported_seq":5343,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960715+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:02:44.960715+0000","last_peered":"2024-09-22T01:02:44.960715+0000","last_clean":"2024-09-22T01:02:44.960715+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:02:44.960715+0000","last_undegraded":"2024-09-22T01:02:44.960715+0000","last_fullsized":"2024-09-22T01:02:44.960715+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:49.738379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":39623057,"num_objects":65,"num_object_clones":21,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":6994,"num_read_kb":28695,"num_write":813,"num_write_kb":17606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"118'400","reported_seq":4468,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960423+0000","last_change":"2024-09-22T01:02:38.741881+0000","last_active":"2024-09-22T01:02:44.960423+0000","last_peered":"2024-09-22T01:02:44.960423+0000","last_clean":"2024-09-22T01:02:44.960423+0000","last_became_active":"2024-09-22T01:02:38.737922+0000","last_became_peered":"2024-09-22T01:02:38.737922+0000","last_unstale":"2024-09-22T01:02:44.960423+0000","last_undegraded":"2024-09-22T01:02:44.960423+0000","last_fullsized":"2024-09-22T01:02:44.960423+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:52.350950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":50428773,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":3758,"num_read_kb":26217,"num_write":469,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":14730069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"118'304","reported_seq":4337,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960179+0000","last_change":"2024-09-22T01:01:43.142453+0000","last_active":"2024-09-22T01:02:44.960179+0000","last_peered":"2024-09-22T01:02:44.960179+0000","last_clean":"2024-09-22T01:02:44.960179+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:02:44.960179+0000","last_undegraded":"2024-09-22T01:02:44.960179+0000","last_fullsized":"2024-09-22T01:02:44.960179+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:48.324524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":30409407,"num_objects":58,"num_object_clones":21,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":26,"num_read":4830,"num_read_kb":10051,"num_write":269,"num_write_kb":12202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"115'357","reported_seq":2803,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.898988+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:02:40.898988+0000","last_peered":"2024-09-22T01:02:40.898988+0000","last_clean":"2024-09-22T01:02:40.898988+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:02:40.898988+0000","last_undegraded":"2024-09-22T01:02:40.898988+0000","last_fullsized":"2024-09-22T01:02:40.898988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:51.389551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":45481595,"num_objects":72,"num_object_clones":22,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":31,"num_read":2986,"num_read_kb":13391,"num_write":335,"num_write_kb":12498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"119'560","reported_seq":2332,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.083269+0000","last_change":"2024-09-22T01:01:43.139843+0000","last_active":"2024-09-22T01:02:45.083269+0000","last_peered":"2024-09-22T01:02:45.083269+0000","last_clean":"2024-09-22T01:02:45.083269+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:02:45.083269+0000","last_undegraded":"2024-09-22T01:02:45.083269+0000","last_fullsized":"2024-09-22T01:02:45.083269+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:27:34.182375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":33886177,"num_objects":52,"num_object_clones":19,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":3997,"num_read_kb":11283,"num_write":731,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"118'361","reported_seq":2694,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961723+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:02:44.961723+0000","last_peered":"2024-09-22T01:02:44.961723+0000","last_clean":"2024-09-22T01:02:44.961723+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:02:44.961723+0000","last_undegraded":"2024-09-22T01:02:44.961723+0000","last_fullsized":"2024-09-22T01:02:44.961723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:54.959321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":26774974,"num_objects":59,"num_object_clones":21,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":28,"num_read":3252,"num_read_kb":7271,"num_write":366,"num_write_kb":8136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"119'274","reported_seq":6264,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990312+0000","last_change":"2024-09-22T01:02:06.801024+0000","last_active":"2024-09-22T01:02:44.990312+0000","last_peered":"2024-09-22T01:02:44.990312+0000","last_clean":"2024-09-22T01:02:44.990312+0000","last_became_active":"2024-09-22T01:02:06.799165+0000","last_became_peered":"2024-09-22T01:02:06.799165+0000","last_unstale":"2024-09-22T01:02:44.990312+0000","last_undegraded":"2024-09-22T01:02:44.990312+0000","last_fullsized":"2024-09-22T01:02:44.990312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:02:44.962116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":30449431,"num_objects":47,"num_object_clones":16,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5691,"num_read_kb":26869,"num_write":330,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":11755308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1540264693,"num_objects":1922,"num_object_clones":586,"num_object_copies":3844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1922,"num_whiteouts":744,"num_read":190799,"num_read_kb":746990,"num_write":20325,"num_write_kb":409209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2782,"num_bytes_recovered":175106004,"num_keys_recovered":760,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":556691456,"data_stored":660107862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":16271,"ondisk_log_size":16271,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103940,"kb_used_data":60172,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94267900,"statfs":{"total":96636764160,"available":96530329600,"internally_reserved":0,"allocated":61616128,"data_stored":71664548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299408,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133504,"kb_used_data":92860,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94238336,"statfs":{"total":96636764160,"available":96500056064,"internally_reserved":0,"allocated":95088640,"data_stored":121536302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":41545294},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130196,"kb_used_data":76800,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94241644,"statfs":{"total":96636764160,"available":96503443456,"internally_reserved":0,"allocated":78643200,"data_stored":90318886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542175,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150040,"kb_used_data":92476,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94221800,"statfs":{"total":96636764160,"available":96483123200,"internally_reserved":0,"allocated":94695424,"data_stored":111271363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74332,"kb_used_data":43136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297508,"statfs":{"total":96636764160,"available":96560648192,"internally_reserved":0,"allocated":44171264,"data_stored":53338065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133236,"kb_used_data":75672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94238604,"statfs":{"total":96636764160,"available":96500330496,"internally_reserved":0,"allocated":77488128,"data_stored":88854456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145264,"kb_used_data":87704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94226576,"statfs":{"total":96636764160,"available":96488013824,"internally_reserved":0,"allocated":89808896,"data_stored":98581825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87801856,"data_stored":97327426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":76070912,"data_stored":88190406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42758144,"data_stored":52674034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":93282304,"data_stored":110607332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":93167616,"data_stored":120340450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":60252160,"data_stored":71024095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":76566528,"data_stored":89008122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:02:46.202 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:02:46.209 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:46.209 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:46.223 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:46.242 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:46.241+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:46.334 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:46.333+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:46.389 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:46.389 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:46.403 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:46.449 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:46.445+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:46.520 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:46.520 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:46.534 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:46.552 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:02:46.570 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-22T01:02:45.794545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1540855080,"num_objects":1926,"num_object_clones":586,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":744,"num_read":190915,"num_read_kb":747090,"num_write":20486,"num_write_kb":411627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":175106042,"num_keys_recovered":760,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16391,"ondisk_log_size":16391,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":870512,"kb_used_data":528820,"kb_used_omap":13,"kb_used_meta":341362,"kb_avail":659732368,"statfs":{"total":676457349120,"available":675565944832,"internally_reserved":0,"allocated":541511680,"data_stored":635565445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":349554932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-885071,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3945,"num_read_kb":7718,"num_write":1035,"num_write_kb":5655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031702"},"pg_stats":[{"pgid":"3.1f","version":"118'440","reported_seq":6708,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961841+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:02:44.961841+0000","last_peered":"2024-09-22T01:02:44.961841+0000","last_clean":"2024-09-22T01:02:44.961841+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:02:44.961841+0000","last_undegraded":"2024-09-22T01:02:44.961841+0000","last_fullsized":"2024-09-22T01:02:44.961841+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:38.421694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":72398686,"num_objects":68,"num_object_clones":21,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":19,"num_read":6471,"num_read_kb":45914,"num_write":501,"num_write_kb":17789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"119'640","reported_seq":2950,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.083658+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:02:45.083658+0000","last_peered":"2024-09-22T01:02:45.083658+0000","last_clean":"2024-09-22T01:02:45.083658+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:02:45.083658+0000","last_undegraded":"2024-09-22T01:02:45.083658+0000","last_fullsized":"2024-09-22T01:02:45.083658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:11:31.811392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":28882882,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":5717,"num_read_kb":12441,"num_write":886,"num_write_kb":7177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"115'403","reported_seq":4482,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.959807+0000","last_change":"2024-09-22T01:01:43.138890+0000","last_active":"2024-09-22T01:02:40.959807+0000","last_peered":"2024-09-22T01:02:40.959807+0000","last_clean":"2024-09-22T01:02:40.959807+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:02:40.959807+0000","last_undegraded":"2024-09-22T01:02:40.959807+0000","last_fullsized":"2024-09-22T01:02:40.959807+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:57.315255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83307193,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":28,"num_read":4780,"num_read_kb":31757,"num_write":448,"num_write_kb":17874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"115'956","reported_seq":9662,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.851291+0000","last_change":"2024-09-22T01:02:39.740146+0000","last_active":"2024-09-22T01:02:40.851291+0000","last_peered":"2024-09-22T01:02:40.851291+0000","last_clean":"2024-09-22T01:02:40.851291+0000","last_became_active":"2024-09-22T01:02:39.738124+0000","last_became_peered":"2024-09-22T01:02:39.738124+0000","last_unstale":"2024-09-22T01:02:40.851291+0000","last_undegraded":"2024-09-22T01:02:40.851291+0000","last_fullsized":"2024-09-22T01:02:40.851291+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":956,"log_dups_size":0,"ondisk_log_size":956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:26:18.864536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84842087,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":23,"num_read":11110,"num_read_kb":51376,"num_write":1445,"num_write_kb":18203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":31958297,"num_keys_recovered":274,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"119'987","reported_seq":2321,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.081038+0000","last_change":"2024-09-22T01:01:43.147132+0000","last_active":"2024-09-22T01:02:45.081038+0000","last_peered":"2024-09-22T01:02:45.081038+0000","last_clean":"2024-09-22T01:02:45.081038+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:02:45.081038+0000","last_undegraded":"2024-09-22T01:02:45.081038+0000","last_fullsized":"2024-09-22T01:02:45.081038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":987,"log_dups_size":0,"ondisk_log_size":987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:36:47.660668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":32243249,"num_objects":63,"num_object_clones":24,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":4965,"num_read_kb":15217,"num_write":1461,"num_write_kb":8369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"119'402","reported_seq":11504,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.065778+0000","last_change":"2024-09-22T01:02:39.739952+0000","last_active":"2024-09-22T01:02:45.065778+0000","last_peered":"2024-09-22T01:02:45.065778+0000","last_clean":"2024-09-22T01:02:45.065778+0000","last_became_active":"2024-09-22T01:02:39.738141+0000","last_became_peered":"2024-09-22T01:02:39.738141+0000","last_unstale":"2024-09-22T01:02:45.065778+0000","last_undegraded":"2024-09-22T01:02:45.065778+0000","last_fullsized":"2024-09-22T01:02:45.065778+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:08.091112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":63093539,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":11619,"num_read_kb":25493,"num_write":502,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":7836182,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"118'343","reported_seq":9624,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784426+0000","last_change":"2024-09-22T01:02:37.738989+0000","last_active":"2024-09-22T01:02:44.784426+0000","last_peered":"2024-09-22T01:02:44.784426+0000","last_clean":"2024-09-22T01:02:44.784426+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:02:44.784426+0000","last_undegraded":"2024-09-22T01:02:44.784426+0000","last_fullsized":"2024-09-22T01:02:44.784426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:52:16.421270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":38522760,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":9088,"num_read_kb":31273,"num_write":422,"num_write_kb":9303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"119'635","reported_seq":5845,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990180+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:02:44.990180+0000","last_peered":"2024-09-22T01:02:44.990180+0000","last_clean":"2024-09-22T01:02:44.990180+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:02:44.990180+0000","last_undegraded":"2024-09-22T01:02:44.990180+0000","last_fullsized":"2024-09-22T01:02:44.990180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:01:35.649602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":84264818,"num_objects":71,"num_object_clones":22,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":18,"num_read":7976,"num_read_kb":42180,"num_write":753,"num_write_kb":16828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"119'452","reported_seq":3751,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961897+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:02:44.961897+0000","last_peered":"2024-09-22T01:02:44.961897+0000","last_clean":"2024-09-22T01:02:44.961897+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:02:44.961897+0000","last_undegraded":"2024-09-22T01:02:44.961897+0000","last_fullsized":"2024-09-22T01:02:44.961897+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:32:30.790477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":39597315,"num_objects":63,"num_object_clones":23,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":4515,"num_read_kb":18057,"num_write":572,"num_write_kb":8510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"114'298","reported_seq":1794,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.796009+0000","last_change":"2024-09-22T01:01:27.848516+0000","last_active":"2024-09-22T01:02:40.796009+0000","last_peered":"2024-09-22T01:02:40.796009+0000","last_clean":"2024-09-22T01:02:40.796009+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:02:40.796009+0000","last_undegraded":"2024-09-22T01:02:40.796009+0000","last_fullsized":"2024-09-22T01:02:40.796009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:17.208328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81626369,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":20,"num_read":2564,"num_read_kb":34277,"num_write":357,"num_write_kb":28683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"118'545","reported_seq":6081,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960197+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:02:44.960197+0000","last_peered":"2024-09-22T01:02:44.960197+0000","last_clean":"2024-09-22T01:02:44.960197+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:02:44.960197+0000","last_undegraded":"2024-09-22T01:02:44.960197+0000","last_fullsized":"2024-09-22T01:02:44.960197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:36:59.661355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":33697818,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":8167,"num_read_kb":26512,"num_write":599,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"118'455","reported_seq":13392,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960315+0000","last_change":"2024-09-22T01:01:43.139393+0000","last_active":"2024-09-22T01:02:44.960315+0000","last_peered":"2024-09-22T01:02:44.960315+0000","last_clean":"2024-09-22T01:02:44.960315+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:02:44.960315+0000","last_undegraded":"2024-09-22T01:02:44.960315+0000","last_fullsized":"2024-09-22T01:02:44.960315+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:43:32.338655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":68935816,"num_objects":65,"num_object_clones":24,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":13638,"num_read_kb":40761,"num_write":509,"num_write_kb":17919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":207,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.761521+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:02:40.761521+0000","last_peered":"2024-09-22T01:02:40.761521+0000","last_clean":"2024-09-22T01:02:40.761521+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:02:40.761521+0000","last_undegraded":"2024-09-22T01:02:40.761521+0000","last_fullsized":"2024-09-22T01:02:40.761521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"119'435","reported_seq":2488,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961736+0000","last_change":"2024-09-22T01:02:37.739819+0000","last_active":"2024-09-22T01:02:44.961736+0000","last_peered":"2024-09-22T01:02:44.961736+0000","last_clean":"2024-09-22T01:02:44.961736+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:02:44.961736+0000","last_undegraded":"2024-09-22T01:02:44.961736+0000","last_fullsized":"2024-09-22T01:02:44.961736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:30:12.181033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":21061190,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":3612,"num_read_kb":6109,"num_write":468,"num_write_kb":3304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784723+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:02:44.784723+0000","last_peered":"2024-09-22T01:02:44.784723+0000","last_clean":"2024-09-22T01:02:44.784723+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:02:44.784723+0000","last_undegraded":"2024-09-22T01:02:44.784723+0000","last_fullsized":"2024-09-22T01:02:44.784723+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"118'497","reported_seq":3088,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990292+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:02:44.990292+0000","last_peered":"2024-09-22T01:02:44.990292+0000","last_clean":"2024-09-22T01:02:44.990292+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:02:44.990292+0000","last_undegraded":"2024-09-22T01:02:44.990292+0000","last_fullsized":"2024-09-22T01:02:44.990292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:48:57.425985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":35266930,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":4214,"num_read_kb":12482,"num_write":649,"num_write_kb":6967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960834+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:02:44.960834+0000","last_peered":"2024-09-22T01:02:44.960834+0000","last_clean":"2024-09-22T01:02:44.960834+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:02:44.960834+0000","last_undegraded":"2024-09-22T01:02:44.960834+0000","last_fullsized":"2024-09-22T01:02:44.960834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"118'355","reported_seq":3947,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784621+0000","last_change":"2024-09-22T01:02:38.738598+0000","last_active":"2024-09-22T01:02:44.784621+0000","last_peered":"2024-09-22T01:02:44.784621+0000","last_clean":"2024-09-22T01:02:44.784621+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:02:44.784621+0000","last_undegraded":"2024-09-22T01:02:44.784621+0000","last_fullsized":"2024-09-22T01:02:44.784621+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:47.050434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":40666215,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":30,"num_read":3788,"num_read_kb":26299,"num_write":376,"num_write_kb":16759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784507+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:02:44.784507+0000","last_peered":"2024-09-22T01:02:44.784507+0000","last_clean":"2024-09-22T01:02:44.784507+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:02:44.784507+0000","last_undegraded":"2024-09-22T01:02:44.784507+0000","last_fullsized":"2024-09-22T01:02:44.784507+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"115'502","reported_seq":3661,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.795917+0000","last_change":"2024-09-22T01:01:31.019330+0000","last_active":"2024-09-22T01:02:40.795917+0000","last_peered":"2024-09-22T01:02:40.795917+0000","last_clean":"2024-09-22T01:02:40.795917+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:02:40.795917+0000","last_undegraded":"2024-09-22T01:02:40.795917+0000","last_fullsized":"2024-09-22T01:02:40.795917+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:50.990905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54122868,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":18,"num_read":6107,"num_read_kb":11468,"num_write":466,"num_write_kb":5346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"119'459","reported_seq":2085,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.057741+0000","last_change":"2024-09-22T01:01:43.148315+0000","last_active":"2024-09-22T01:02:45.057741+0000","last_peered":"2024-09-22T01:02:45.057741+0000","last_clean":"2024-09-22T01:02:45.057741+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:02:45.057741+0000","last_undegraded":"2024-09-22T01:02:45.057741+0000","last_fullsized":"2024-09-22T01:02:45.057741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:33:05.221914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":54606605,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":2891,"num_read_kb":17537,"num_write":514,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":180,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.795668+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:02:40.795668+0000","last_peered":"2024-09-22T01:02:40.795668+0000","last_clean":"2024-09-22T01:02:40.795668+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:02:40.795668+0000","last_undegraded":"2024-09-22T01:02:40.795668+0000","last_fullsized":"2024-09-22T01:02:40.795668+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"119'1546","reported_seq":3364,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784518+0000","last_change":"2024-09-22T01:02:39.740320+0000","last_active":"2024-09-22T01:02:44.784518+0000","last_peered":"2024-09-22T01:02:44.784518+0000","last_clean":"2024-09-22T01:02:44.784518+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:02:44.784518+0000","last_undegraded":"2024-09-22T01:02:44.784518+0000","last_fullsized":"2024-09-22T01:02:44.784518+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1546,"log_dups_size":0,"ondisk_log_size":1546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:56.118200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":49656345,"num_objects":67,"num_object_clones":23,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":8866,"num_read_kb":25283,"num_write":2641,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784066+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:02:44.784066+0000","last_peered":"2024-09-22T01:02:44.784066+0000","last_clean":"2024-09-22T01:02:44.784066+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:02:44.784066+0000","last_undegraded":"2024-09-22T01:02:44.784066+0000","last_fullsized":"2024-09-22T01:02:44.784066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'339","reported_seq":2504,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961699+0000","last_change":"2024-09-22T01:01:43.152419+0000","last_active":"2024-09-22T01:02:44.961699+0000","last_peered":"2024-09-22T01:02:44.961699+0000","last_clean":"2024-09-22T01:02:44.961699+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:02:44.961699+0000","last_undegraded":"2024-09-22T01:02:44.961699+0000","last_fullsized":"2024-09-22T01:02:44.961699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:57.141863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":44140547,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":2187,"num_read_kb":18579,"num_write":371,"num_write_kb":15217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":226,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960134+0000","last_change":"2024-09-22T01:02:35.049954+0000","last_active":"2024-09-22T01:02:44.960134+0000","last_peered":"2024-09-22T01:02:44.960134+0000","last_clean":"2024-09-22T01:02:44.960134+0000","last_became_active":"2024-09-22T01:02:35.049024+0000","last_became_peered":"2024-09-22T01:02:35.049024+0000","last_unstale":"2024-09-22T01:02:44.960134+0000","last_undegraded":"2024-09-22T01:02:44.960134+0000","last_fullsized":"2024-09-22T01:02:44.960134+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:45.992332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"118'328","reported_seq":1932,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990212+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:02:44.990212+0000","last_peered":"2024-09-22T01:02:44.990212+0000","last_clean":"2024-09-22T01:02:44.990212+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:02:44.990212+0000","last_undegraded":"2024-09-22T01:02:44.990212+0000","last_fullsized":"2024-09-22T01:02:44.990212+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:55.260294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":36976643,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":19,"num_read":1621,"num_read_kb":15577,"num_write":344,"num_write_kb":17862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.795673+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:02:40.795673+0000","last_peered":"2024-09-22T01:02:40.795673+0000","last_clean":"2024-09-22T01:02:40.795673+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:02:40.795673+0000","last_undegraded":"2024-09-22T01:02:40.795673+0000","last_fullsized":"2024-09-22T01:02:40.795673+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":367,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.959934+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:02:44.959934+0000","last_peered":"2024-09-22T01:02:44.959934+0000","last_clean":"2024-09-22T01:02:44.959934+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:02:44.959934+0000","last_undegraded":"2024-09-22T01:02:44.959934+0000","last_fullsized":"2024-09-22T01:02:44.959934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"119'360","reported_seq":10029,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990192+0000","last_change":"2024-09-22T01:02:39.736794+0000","last_active":"2024-09-22T01:02:44.990192+0000","last_peered":"2024-09-22T01:02:44.990192+0000","last_clean":"2024-09-22T01:02:44.990192+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:02:44.990192+0000","last_undegraded":"2024-09-22T01:02:44.990192+0000","last_fullsized":"2024-09-22T01:02:44.990192+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:53:34.004734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":60097571,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":28,"num_read":10369,"num_read_kb":27110,"num_write":380,"num_write_kb":13691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"119'491","reported_seq":1921,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.038837+0000","last_change":"2024-09-22T01:01:43.142999+0000","last_active":"2024-09-22T01:02:45.038837+0000","last_peered":"2024-09-22T01:02:45.038837+0000","last_clean":"2024-09-22T01:02:45.038837+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:02:45.038837+0000","last_undegraded":"2024-09-22T01:02:45.038837+0000","last_fullsized":"2024-09-22T01:02:45.038837+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:16:43.137041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":41881935,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":21,"num_read":2593,"num_read_kb":6970,"num_write":588,"num_write_kb":8014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"115'613","reported_seq":15132,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.761511+0000","last_change":"2024-09-22T01:02:37.738638+0000","last_active":"2024-09-22T01:02:40.761511+0000","last_peered":"2024-09-22T01:02:40.761511+0000","last_clean":"2024-09-22T01:02:40.761511+0000","last_became_active":"2024-09-22T01:02:37.737218+0000","last_became_peered":"2024-09-22T01:02:37.737218+0000","last_unstale":"2024-09-22T01:02:40.761511+0000","last_undegraded":"2024-09-22T01:02:40.761511+0000","last_fullsized":"2024-09-22T01:02:40.761511+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:06:56.183242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057495099999999998,"stat_sum":{"num_bytes":62538447,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":21,"num_read":15926,"num_read_kb":39962,"num_write":677,"num_write_kb":12813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":13401696,"num_keys_recovered":167,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"119'565","reported_seq":2256,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.039730+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:02:45.039730+0000","last_peered":"2024-09-22T01:02:45.039730+0000","last_clean":"2024-09-22T01:02:45.039730+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:02:45.039730+0000","last_undegraded":"2024-09-22T01:02:45.039730+0000","last_fullsized":"2024-09-22T01:02:45.039730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:11.769709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":22105578,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":3955,"num_read_kb":19562,"num_write":725,"num_write_kb":10845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"115'336","reported_seq":1001,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.795994+0000","last_change":"2024-09-22T01:01:27.940596+0000","last_active":"2024-09-22T01:02:40.795994+0000","last_peered":"2024-09-22T01:02:40.795994+0000","last_clean":"2024-09-22T01:02:40.795994+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:02:40.795994+0000","last_undegraded":"2024-09-22T01:02:40.795994+0000","last_fullsized":"2024-09-22T01:02:40.795994+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:24.542838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48677873,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":2552,"num_read_kb":21017,"num_write":358,"num_write_kb":10863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"119'633","reported_seq":5343,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960715+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:02:44.960715+0000","last_peered":"2024-09-22T01:02:44.960715+0000","last_clean":"2024-09-22T01:02:44.960715+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:02:44.960715+0000","last_undegraded":"2024-09-22T01:02:44.960715+0000","last_fullsized":"2024-09-22T01:02:44.960715+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:49.738379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":39623057,"num_objects":65,"num_object_clones":21,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":6994,"num_read_kb":28695,"num_write":813,"num_write_kb":17606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"118'400","reported_seq":4468,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960423+0000","last_change":"2024-09-22T01:02:38.741881+0000","last_active":"2024-09-22T01:02:44.960423+0000","last_peered":"2024-09-22T01:02:44.960423+0000","last_clean":"2024-09-22T01:02:44.960423+0000","last_became_active":"2024-09-22T01:02:38.737922+0000","last_became_peered":"2024-09-22T01:02:38.737922+0000","last_unstale":"2024-09-22T01:02:44.960423+0000","last_undegraded":"2024-09-22T01:02:44.960423+0000","last_fullsized":"2024-09-22T01:02:44.960423+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:52.350950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":50428773,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":3758,"num_read_kb":26217,"num_write":469,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":14730069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"118'304","reported_seq":4337,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960179+0000","last_change":"2024-09-22T01:01:43.142453+0000","last_active":"2024-09-22T01:02:44.960179+0000","last_peered":"2024-09-22T01:02:44.960179+0000","last_clean":"2024-09-22T01:02:44.960179+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:02:44.960179+0000","last_undegraded":"2024-09-22T01:02:44.960179+0000","last_fullsized":"2024-09-22T01:02:44.960179+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:48.324524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":30409407,"num_objects":58,"num_object_clones":21,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":26,"num_read":4830,"num_read_kb":10051,"num_write":269,"num_write_kb":12202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"115'357","reported_seq":2803,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-22T01:02:40.898988+0000","last_change":"2024-09-22T01:00:49.708519+0000","last_active":"2024-09-22T01:02:40.898988+0000","last_peered":"2024-09-22T01:02:40.898988+0000","last_clean":"2024-09-22T01:02:40.898988+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:02:40.898988+0000","last_undegraded":"2024-09-22T01:02:40.898988+0000","last_fullsized":"2024-09-22T01:02:40.898988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:51.389551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045816999999999999,"stat_sum":{"num_bytes":45481595,"num_objects":72,"num_object_clones":22,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":31,"num_read":2986,"num_read_kb":13391,"num_write":335,"num_write_kb":12498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"119'560","reported_seq":2332,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.083269+0000","last_change":"2024-09-22T01:01:43.139843+0000","last_active":"2024-09-22T01:02:45.083269+0000","last_peered":"2024-09-22T01:02:45.083269+0000","last_clean":"2024-09-22T01:02:45.083269+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:02:45.083269+0000","last_undegraded":"2024-09-22T01:02:45.083269+0000","last_fullsized":"2024-09-22T01:02:45.083269+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:27:34.182375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":33886177,"num_objects":52,"num_object_clones":19,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":3997,"num_read_kb":11283,"num_write":731,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"118'361","reported_seq":2694,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961723+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:02:44.961723+0000","last_peered":"2024-09-22T01:02:44.961723+0000","last_clean":"2024-09-22T01:02:44.961723+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:02:44.961723+0000","last_undegraded":"2024-09-22T01:02:44.961723+0000","last_fullsized":"2024-09-22T01:02:44.961723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:54.959321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":26774974,"num_objects":59,"num_object_clones":21,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":28,"num_read":3252,"num_read_kb":7271,"num_write":366,"num_write_kb":8136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"119'274","reported_seq":6264,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990312+0000","last_change":"2024-09-22T01:02:06.801024+0000","last_active":"2024-09-22T01:02:44.990312+0000","last_peered":"2024-09-22T01:02:44.990312+0000","last_clean":"2024-09-22T01:02:44.990312+0000","last_became_active":"2024-09-22T01:02:06.799165+0000","last_became_peered":"2024-09-22T01:02:06.799165+0000","last_unstale":"2024-09-22T01:02:44.990312+0000","last_undegraded":"2024-09-22T01:02:44.990312+0000","last_fullsized":"2024-09-22T01:02:44.990312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:02:44.962116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":30449431,"num_objects":47,"num_object_clones":16,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5691,"num_read_kb":26869,"num_write":330,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":11755308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1540264693,"num_objects":1922,"num_object_clones":586,"num_object_copies":3844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1922,"num_whiteouts":744,"num_read":190799,"num_read_kb":746990,"num_write":20325,"num_write_kb":409209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2782,"num_bytes_recovered":175106004,"num_keys_recovered":760,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":556691456,"data_stored":660107862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":16271,"ondisk_log_size":16271,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103940,"kb_used_data":60172,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94267900,"statfs":{"total":96636764160,"available":96530329600,"internally_reserved":0,"allocated":61616128,"data_stored":71664548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299408,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133504,"kb_used_data":92860,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94238336,"statfs":{"total":96636764160,"available":96500056064,"internally_reserved":0,"allocated":95088640,"data_stored":121536302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":41545294},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130196,"kb_used_data":76800,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94241644,"statfs":{"total":96636764160,"available":96503443456,"internally_reserved":0,"allocated":78643200,"data_stored":90318886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542175,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150040,"kb_used_data":92476,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94221800,"statfs":{"total":96636764160,"available":96483123200,"internally_reserved":0,"allocated":94695424,"data_stored":111271363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74332,"kb_used_data":43136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297508,"statfs":{"total":96636764160,"available":96560648192,"internally_reserved":0,"allocated":44171264,"data_stored":53338065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133236,"kb_used_data":75672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94238604,"statfs":{"total":96636764160,"available":96500330496,"internally_reserved":0,"allocated":77488128,"data_stored":88854456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145264,"kb_used_data":87704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94226576,"statfs":{"total":96636764160,"available":96488013824,"internally_reserved":0,"allocated":89808896,"data_stored":98581825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87801856,"data_stored":97327426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":76070912,"data_stored":88190406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42758144,"data_stored":52674034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":93282304,"data_stored":110607332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":93167616,"data_stored":120340450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":60252160,"data_stored":71024095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":76566528,"data_stored":89008122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:02:46.572 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T01:02:46.572 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:02:46.573 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-22T01:02:46.573 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:02:46.577 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:46.573+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:46.637 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:46.629+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:46.651 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:46.651 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:46.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:46.758 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:46.757+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:46.803 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:02:46.838 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:46.833+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:46.842 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:46.842 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:46.859 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:46.942 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 121, 2024-09-22T01:02:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:02:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-22T01:00:18.015746+0000", 2024-09-22T01:02:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-22T01:02:46.793421+0000", 2024-09-22T01:02:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-22T01:01:26.960327+0000", 2024-09-22T01:02:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-22T01:02:34.026131+0000", 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5801984, 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "noscrub", 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:02:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:30.699559+0000", 2024-09-22T01:02:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-22T01:02:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-22T01:02:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-22T01:02:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:36.410605+0000", 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 2, 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 21, 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9900000095367432, 2024-09-22T01:02:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9900000095367432, 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:46.342370+0000", 2024-09-22T01:02:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-22T01:02:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "121", 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 50, 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 121, 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.7400000095367432, 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.7400000095367432, 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:02:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "8c0f94a8-3c63-442d-9dc1-50c1278929da", 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 112, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2277223948", 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2277223948", 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2277223948", 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2277223948", 2024-09-22T01:02:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "2d2cf30e-e9bd-40ba-8293-193faa02ea69", 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 113, 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-22T01:02:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-22T01:02:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-22T01:02:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/463340194", 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/463340194", 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/463340194", 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/463340194", 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a3d3e330-9ea1-4a5c-811b-f285f9a3cff4", 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 112, 2024-09-22T01:02:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-22T01:02:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-22T01:02:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-22T01:02:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/387643397", 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/387643397", 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/387643397", 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/387643397", 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "239072f0-4b69-4624-ba6c-733f299208da", 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 113, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1215210764", 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1215210764", 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1215210764", 2024-09-22T01:02:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1215210764", 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fd35cb24-eb2c-4c17-abe9-ce5c34db9014", 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 0, 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0, 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 54, 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6800", 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6801", 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6802", 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6803", 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6806", 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6807", 2024-09-22T01:02:46.989 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6804", 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.990 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6805", 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6801/2463696152", 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6803/2463696152", 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6807/2463696152", 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6805/2463696152", 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "b60a584d-5334-41e4-92d9-f6689de6b8bf", 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 15, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 25, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 52, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 113, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 26, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:46.992 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6824", 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6825", 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:46.993 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6826", 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6827", 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.994 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6830", 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6831", 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.995 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6828", 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6829", 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.996 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6825/1737475131", 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6827/1737475131", 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1737475131", 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1737475131", 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "9be24894-1100-4e4e-bf1b-79e163fd97e0", 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:46.997 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 111, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.998 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6816", 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6817", 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:46.999 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6818", 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6819", 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.000 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6822", 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6823", 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.001 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6820", 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6821", 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6817/4220217922", 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6819/4220217922", 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6823/4220217922", 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6821/4220217922", 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "293cfbc3-8c2e-40b5-ab89-0a124cc8651c", 2024-09-22T01:02:47.003 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 112, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6808", 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6809", 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:47.005 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6810", 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6811", 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:47.006 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6814", 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6815", 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6812", 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6813", 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6809/3867156106", 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6811/3867156106", 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3867156106", 2024-09-22T01:02:47.008 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3867156106", 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:47.009 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.556128+0000", 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.159753+0000", 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.766199+0000", 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:47.011 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.155529+0000", 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 65536, 2024-09-22T01:02:47.012 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.226232+0000", 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.243346+0000", 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 26 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:02:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.143084+0000", 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:47.014 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.524369+0000", 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [ 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.13", 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 7, 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 3 2024-09-22T01:02:47.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-22T01:02:47.016 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [ 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 49, 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 2 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.017 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [ 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 49, 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 1 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.019 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:47.020 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:46.941+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:47.021 INFO:tasks.thrashosds.thrasher:j is {'epoch': 121, 'fsid': '7d08b428-4185-4f0f-a0fc-f7dc1fefdce9', 'created': '2024-09-22T01:00:18.015746+0000', 'modified': '2024-09-22T01:02:46.793421+0000', 'last_up_change': '2024-09-22T01:01:26.960327+0000', 'last_in_change': '2024-09-22T01:02:34.026131+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:00:30.699559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:00:36.410605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:00:46.342370+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8c0f94a8-3c63-442d-9dc1-50c1278929da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2277223948}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2277223948}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2277223948}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2277223948}]}, 'public_addr': '172.21.15.2:6825/2277223948', 'cluster_addr': '172.21.15.2:6827/2277223948', 'heartbeat_back_addr': '172.21.15.2:6831/2277223948', 'heartbeat_front_addr': '172.21.15.2:6829/2277223948', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 463340194}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 463340194}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 463340194}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 463340194}]}, 'public_addr': '172.21.15.2:6817/463340194', 'cluster_addr': '172.21.15.2:6819/463340194', 'heartbeat_back_addr': '172.21.15.2:6823/463340194', 'heartbeat_front_addr': '172.21.15.2:6821/463340194', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 387643397}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 387643397}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 387643397}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 387643397}]}, 'public_addr': '172.21.15.2:6801/387643397', 'cluster_addr': '172.21.15.2:6803/387643397', 'heartbeat_back_addr': '172.21.15.2:6807/387643397', 'heartbeat_front_addr': '172.21.15.2:6805/387643397', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '239072f0-4b69-4624-ba6c-733f299208da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1215210764}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1215210764}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1215210764}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1215210764}]}, 'public_addr': '172.21.15.2:6809/1215210764', 'cluster_addr': '172.21.15.2:6811/1215210764', 'heartbeat_back_addr': '172.21.15.2:6815/1215210764', 'heartbeat_front_addr': '172.21.15.2:6813/1215210764', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 2463696152}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 2463696152}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 2463696152}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 2463696152}]}, 'public_addr': '172.21.15.119:6801/2463696152', 'cluster_addr': '172.21.15.119:6803/2463696152', 'heartbeat_back_addr': '172.21.15.119:6807/2463696152', 'heartbeat_front_addr': '172.21.15.119:6805/2463696152', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b60a584d-5334-41e4-92d9-f6689de6b8bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 25, 'up_from': 52, 'up_thru': 113, 'down_at': 26, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1737475131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1737475131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1737475131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1737475131}]}, 'public_addr': '172.21.15.119:6825/1737475131', 'cluster_addr': '172.21.15.119:6827/1737475131', 'heartbeat_back_addr': '172.21.15.119:6831/1737475131', 'heartbeat_front_addr': '172.21.15.119:6829/1737475131', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9be24894-1100-4e4e-bf1b-79e163fd97e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 4220217922}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 4220217922}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 4220217922}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 4220217922}]}, 'public_addr': '172.21.15.119:6817/4220217922', 'cluster_addr': '172.21.15.119:6819/4220217922', 'heartbeat_back_addr': '172.21.15.119:6823/4220217922', 'heartbeat_front_addr': '172.21.15.119:6821/4220217922', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '293cfbc3-8c2e-40b5-ab89-0a124cc8651c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3867156106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3867156106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3867156106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3867156106}]}, 'public_addr': '172.21.15.119:6809/3867156106', 'cluster_addr': '172.21.15.119:6811/3867156106', 'heartbeat_back_addr': '172.21.15.119:6815/3867156106', 'heartbeat_front_addr': '172.21.15.119:6813/3867156106', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.556128+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.159753+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.766199+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.155529+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.226232+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.243346+0000', 'dead_epoch': 26}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.143084+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.524369+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.13', 'mappings': [{'from': 7, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 49, 'length': 2}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 49, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:02:47.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:02:47.026 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:47.026 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:47.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:47.041 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:47.037+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:47.077 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:47.139 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:47.133+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:47.208 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:47.208 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:47.224 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:47.239 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:47.239 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:47.240 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:47.233+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:47.257 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:47.274 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:47.291 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:47.291 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:47.311 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:47.312 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:47.332 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:47.332 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:47.334 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:47.335 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:47.335 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:47.335 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:47.335 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:47.339 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:02:47.341 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:47.337+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:47.350 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:47.353 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:47.354 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:47.358 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-22T01:02:46.801857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1538814405,"num_objects":1916,"num_object_clones":586,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":744,"num_read":190974,"num_read_kb":749956,"num_write":20591,"num_write_kb":412946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":175106042,"num_keys_recovered":760,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16493,"ondisk_log_size":16493,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":852332,"kb_used_data":527420,"kb_used_omap":13,"kb_used_meta":324722,"kb_avail":659750548,"statfs":{"total":676457349120,"available":675584561152,"internally_reserved":0,"allocated":540078080,"data_stored":634087436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":332515572},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2925746,"num_objects":-7,"num_object_clones":0,"num_object_copies":-14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7,"num_whiteouts":0,"num_read":4004,"num_read_kb":10584,"num_write":1140,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.032218"},"pg_stats":[{"pgid":"3.1f","version":"118'440","reported_seq":6708,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961841+0000","last_change":"2024-09-22T01:01:31.021247+0000","last_active":"2024-09-22T01:02:44.961841+0000","last_peered":"2024-09-22T01:02:44.961841+0000","last_clean":"2024-09-22T01:02:44.961841+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:02:44.961841+0000","last_undegraded":"2024-09-22T01:02:44.961841+0000","last_fullsized":"2024-09-22T01:02:44.961841+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:38.421694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002115399,"stat_sum":{"num_bytes":72398686,"num_objects":68,"num_object_clones":21,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":19,"num_read":6471,"num_read_kb":45914,"num_write":501,"num_write_kb":17789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"119'640","reported_seq":2950,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.083658+0000","last_change":"2024-09-22T01:00:49.736410+0000","last_active":"2024-09-22T01:02:45.083658+0000","last_peered":"2024-09-22T01:02:45.083658+0000","last_clean":"2024-09-22T01:02:45.083658+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:02:45.083658+0000","last_undegraded":"2024-09-22T01:02:45.083658+0000","last_fullsized":"2024-09-22T01:02:45.083658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:11:31.811392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001286113,"stat_sum":{"num_bytes":28882882,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":5717,"num_read_kb":12441,"num_write":886,"num_write_kb":7177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"120'418","reported_seq":4517,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.912804+0000","last_change":"2024-09-22T01:02:45.799258+0000","last_active":"2024-09-22T01:02:45.912804+0000","last_peered":"2024-09-22T01:02:45.912804+0000","last_clean":"2024-09-22T01:02:45.912804+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:02:45.912804+0000","last_undegraded":"2024-09-22T01:02:45.912804+0000","last_fullsized":"2024-09-22T01:02:45.912804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:18:33.170517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001511882,"stat_sum":{"num_bytes":82705311,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":28,"num_read":4784,"num_read_kb":32327,"num_write":463,"num_write_kb":18066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.1c","version":"120'967","reported_seq":9704,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.855322+0000","last_change":"2024-09-22T01:02:45.799252+0000","last_active":"2024-09-22T01:02:45.855322+0000","last_peered":"2024-09-22T01:02:45.855322+0000","last_clean":"2024-09-22T01:02:45.855322+0000","last_became_active":"2024-09-22T01:02:39.738124+0000","last_became_peered":"2024-09-22T01:02:39.738124+0000","last_unstale":"2024-09-22T01:02:45.855322+0000","last_undegraded":"2024-09-22T01:02:45.855322+0000","last_fullsized":"2024-09-22T01:02:45.855322+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:58:46.271586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953369999999999,"stat_sum":{"num_bytes":84842087,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":23,"num_read":11136,"num_read_kb":51394,"num_write":1458,"num_write_kb":18274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":31958297,"num_keys_recovered":274,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.1b","version":"119'987","reported_seq":2321,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.081038+0000","last_change":"2024-09-22T01:01:43.147132+0000","last_active":"2024-09-22T01:02:45.081038+0000","last_peered":"2024-09-22T01:02:45.081038+0000","last_clean":"2024-09-22T01:02:45.081038+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:02:45.081038+0000","last_undegraded":"2024-09-22T01:02:45.081038+0000","last_fullsized":"2024-09-22T01:02:45.081038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":987,"log_dups_size":0,"ondisk_log_size":987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:36:47.660668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180232,"stat_sum":{"num_bytes":32243249,"num_objects":63,"num_object_clones":24,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":4965,"num_read_kb":15217,"num_write":1461,"num_write_kb":8369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"119'402","reported_seq":11504,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.065778+0000","last_change":"2024-09-22T01:02:39.739952+0000","last_active":"2024-09-22T01:02:45.065778+0000","last_peered":"2024-09-22T01:02:45.065778+0000","last_clean":"2024-09-22T01:02:45.065778+0000","last_became_active":"2024-09-22T01:02:39.738141+0000","last_became_peered":"2024-09-22T01:02:39.738141+0000","last_unstale":"2024-09-22T01:02:45.065778+0000","last_undegraded":"2024-09-22T01:02:45.065778+0000","last_fullsized":"2024-09-22T01:02:45.065778+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:08.091112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016171639999999999,"stat_sum":{"num_bytes":63093539,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":11619,"num_read_kb":25493,"num_write":502,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":7836182,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"118'343","reported_seq":9624,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784426+0000","last_change":"2024-09-22T01:02:37.738989+0000","last_active":"2024-09-22T01:02:44.784426+0000","last_peered":"2024-09-22T01:02:44.784426+0000","last_clean":"2024-09-22T01:02:44.784426+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:02:44.784426+0000","last_undegraded":"2024-09-22T01:02:44.784426+0000","last_fullsized":"2024-09-22T01:02:44.784426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:52:16.421270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065649899999999995,"stat_sum":{"num_bytes":38522760,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":9088,"num_read_kb":31273,"num_write":422,"num_write_kb":9303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"119'635","reported_seq":5845,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990180+0000","last_change":"2024-09-22T01:00:49.735654+0000","last_active":"2024-09-22T01:02:44.990180+0000","last_peered":"2024-09-22T01:02:44.990180+0000","last_clean":"2024-09-22T01:02:44.990180+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:02:44.990180+0000","last_undegraded":"2024-09-22T01:02:44.990180+0000","last_fullsized":"2024-09-22T01:02:44.990180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:01:35.649602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067980500000000001,"stat_sum":{"num_bytes":84264818,"num_objects":71,"num_object_clones":22,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":18,"num_read":7976,"num_read_kb":42180,"num_write":753,"num_write_kb":16828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"119'452","reported_seq":3751,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961897+0000","last_change":"2024-09-22T01:01:27.803857+0000","last_active":"2024-09-22T01:02:44.961897+0000","last_peered":"2024-09-22T01:02:44.961897+0000","last_clean":"2024-09-22T01:02:44.961897+0000","last_became_active":"2024-09-22T01:01:27.731561+0000","last_became_peered":"2024-09-22T01:01:27.731561+0000","last_unstale":"2024-09-22T01:02:44.961897+0000","last_undegraded":"2024-09-22T01:02:44.961897+0000","last_fullsized":"2024-09-22T01:02:44.961897+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:32:30.790477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019324769999999999,"stat_sum":{"num_bytes":39597315,"num_objects":63,"num_object_clones":23,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":4515,"num_read_kb":18057,"num_write":572,"num_write_kb":8510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":601668,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"118'309","reported_seq":1821,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.797887+0000","last_change":"2024-09-22T01:02:45.798442+0000","last_active":"2024-09-22T01:02:45.797887+0000","last_peered":"2024-09-22T01:02:45.797887+0000","last_clean":"2024-09-22T01:02:45.797887+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:02:45.797887+0000","last_undegraded":"2024-09-22T01:02:45.797887+0000","last_fullsized":"2024-09-22T01:02:45.797887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:38:49.774766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010521739999999999,"stat_sum":{"num_bytes":81626369,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":20,"num_read":2565,"num_read_kb":34277,"num_write":367,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.15","version":"118'545","reported_seq":6081,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960197+0000","last_change":"2024-09-22T01:00:49.863495+0000","last_active":"2024-09-22T01:02:44.960197+0000","last_peered":"2024-09-22T01:02:44.960197+0000","last_clean":"2024-09-22T01:02:44.960197+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:02:44.960197+0000","last_undegraded":"2024-09-22T01:02:44.960197+0000","last_fullsized":"2024-09-22T01:02:44.960197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:36:59.661355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050219800000000003,"stat_sum":{"num_bytes":33697818,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":8167,"num_read_kb":26512,"num_write":599,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"118'455","reported_seq":13392,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960315+0000","last_change":"2024-09-22T01:01:43.139393+0000","last_active":"2024-09-22T01:02:44.960315+0000","last_peered":"2024-09-22T01:02:44.960315+0000","last_clean":"2024-09-22T01:02:44.960315+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:02:44.960315+0000","last_undegraded":"2024-09-22T01:02:44.960315+0000","last_fullsized":"2024-09-22T01:02:44.960315+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:43:32.338655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090090299999999999,"stat_sum":{"num_bytes":68935816,"num_objects":65,"num_object_clones":24,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":13638,"num_read_kb":40761,"num_write":509,"num_write_kb":17919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":217,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.811654+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:02:45.811654+0000","last_peered":"2024-09-22T01:02:45.811654+0000","last_clean":"2024-09-22T01:02:45.811654+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:02:45.811654+0000","last_undegraded":"2024-09-22T01:02:45.811654+0000","last_fullsized":"2024-09-22T01:02:45.811654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"119'435","reported_seq":2488,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961736+0000","last_change":"2024-09-22T01:02:37.739819+0000","last_active":"2024-09-22T01:02:44.961736+0000","last_peered":"2024-09-22T01:02:44.961736+0000","last_clean":"2024-09-22T01:02:44.961736+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:02:44.961736+0000","last_undegraded":"2024-09-22T01:02:44.961736+0000","last_fullsized":"2024-09-22T01:02:44.961736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:30:12.181033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065639099999999998,"stat_sum":{"num_bytes":21061190,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":3612,"num_read_kb":6109,"num_write":468,"num_write_kb":3304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784723+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:02:44.784723+0000","last_peered":"2024-09-22T01:02:44.784723+0000","last_clean":"2024-09-22T01:02:44.784723+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:02:44.784723+0000","last_undegraded":"2024-09-22T01:02:44.784723+0000","last_fullsized":"2024-09-22T01:02:44.784723+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"118'497","reported_seq":3088,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990292+0000","last_change":"2024-09-22T01:00:49.736778+0000","last_active":"2024-09-22T01:02:44.990292+0000","last_peered":"2024-09-22T01:02:44.990292+0000","last_clean":"2024-09-22T01:02:44.990292+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:02:44.990292+0000","last_undegraded":"2024-09-22T01:02:44.990292+0000","last_fullsized":"2024-09-22T01:02:44.990292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:48:57.425985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361608,"stat_sum":{"num_bytes":35266930,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":4214,"num_read_kb":12482,"num_write":649,"num_write_kb":6967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960834+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:02:44.960834+0000","last_peered":"2024-09-22T01:02:44.960834+0000","last_clean":"2024-09-22T01:02:44.960834+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:02:44.960834+0000","last_undegraded":"2024-09-22T01:02:44.960834+0000","last_fullsized":"2024-09-22T01:02:44.960834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"118'355","reported_seq":3947,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784621+0000","last_change":"2024-09-22T01:02:38.738598+0000","last_active":"2024-09-22T01:02:44.784621+0000","last_peered":"2024-09-22T01:02:44.784621+0000","last_clean":"2024-09-22T01:02:44.784621+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:02:44.784621+0000","last_undegraded":"2024-09-22T01:02:44.784621+0000","last_fullsized":"2024-09-22T01:02:44.784621+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:47.050434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010114569999999999,"stat_sum":{"num_bytes":40666215,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":30,"num_read":3788,"num_read_kb":26299,"num_write":376,"num_write_kb":16759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784507+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:02:44.784507+0000","last_peered":"2024-09-22T01:02:44.784507+0000","last_clean":"2024-09-22T01:02:44.784507+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:02:44.784507+0000","last_undegraded":"2024-09-22T01:02:44.784507+0000","last_fullsized":"2024-09-22T01:02:44.784507+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"120'522","reported_seq":3704,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.879303+0000","last_change":"2024-09-22T01:02:45.798904+0000","last_active":"2024-09-22T01:02:45.879303+0000","last_peered":"2024-09-22T01:02:45.879303+0000","last_clean":"2024-09-22T01:02:45.879303+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:02:45.879303+0000","last_undegraded":"2024-09-22T01:02:45.879303+0000","last_fullsized":"2024-09-22T01:02:45.879303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:44.880562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015165160000000001,"stat_sum":{"num_bytes":53860708,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":18,"num_read":6118,"num_read_kb":11982,"num_write":486,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.3","version":"119'459","reported_seq":2085,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.057741+0000","last_change":"2024-09-22T01:01:43.148315+0000","last_active":"2024-09-22T01:02:45.057741+0000","last_peered":"2024-09-22T01:02:45.057741+0000","last_clean":"2024-09-22T01:02:45.057741+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:02:45.057741+0000","last_undegraded":"2024-09-22T01:02:45.057741+0000","last_fullsized":"2024-09-22T01:02:45.057741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:33:05.221914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056016900000000001,"stat_sum":{"num_bytes":54606605,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":2891,"num_read_kb":17537,"num_write":514,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":190,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.796842+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:02:45.796842+0000","last_peered":"2024-09-22T01:02:45.796842+0000","last_clean":"2024-09-22T01:02:45.796842+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:02:45.796842+0000","last_undegraded":"2024-09-22T01:02:45.796842+0000","last_fullsized":"2024-09-22T01:02:45.796842+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"119'1546","reported_seq":3364,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784518+0000","last_change":"2024-09-22T01:02:39.740320+0000","last_active":"2024-09-22T01:02:44.784518+0000","last_peered":"2024-09-22T01:02:44.784518+0000","last_clean":"2024-09-22T01:02:44.784518+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:02:44.784518+0000","last_undegraded":"2024-09-22T01:02:44.784518+0000","last_fullsized":"2024-09-22T01:02:44.784518+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1546,"log_dups_size":0,"ondisk_log_size":1546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:56.118200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048612,"stat_sum":{"num_bytes":49656345,"num_objects":67,"num_object_clones":23,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":8866,"num_read_kb":25283,"num_write":2641,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.784066+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:02:44.784066+0000","last_peered":"2024-09-22T01:02:44.784066+0000","last_clean":"2024-09-22T01:02:44.784066+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:02:44.784066+0000","last_undegraded":"2024-09-22T01:02:44.784066+0000","last_fullsized":"2024-09-22T01:02:44.784066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'339","reported_seq":2504,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961699+0000","last_change":"2024-09-22T01:01:43.152419+0000","last_active":"2024-09-22T01:02:44.961699+0000","last_peered":"2024-09-22T01:02:44.961699+0000","last_clean":"2024-09-22T01:02:44.961699+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:02:44.961699+0000","last_undegraded":"2024-09-22T01:02:44.961699+0000","last_fullsized":"2024-09-22T01:02:44.961699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:57.141863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00145984,"stat_sum":{"num_bytes":44140547,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":2187,"num_read_kb":18579,"num_write":371,"num_write_kb":15217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":226,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960134+0000","last_change":"2024-09-22T01:02:35.049954+0000","last_active":"2024-09-22T01:02:44.960134+0000","last_peered":"2024-09-22T01:02:44.960134+0000","last_clean":"2024-09-22T01:02:44.960134+0000","last_became_active":"2024-09-22T01:02:35.049024+0000","last_became_peered":"2024-09-22T01:02:35.049024+0000","last_unstale":"2024-09-22T01:02:44.960134+0000","last_undegraded":"2024-09-22T01:02:44.960134+0000","last_fullsized":"2024-09-22T01:02:44.960134+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:45.992332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"118'328","reported_seq":1932,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990212+0000","last_change":"2024-09-22T01:01:31.018917+0000","last_active":"2024-09-22T01:02:44.990212+0000","last_peered":"2024-09-22T01:02:44.990212+0000","last_clean":"2024-09-22T01:02:44.990212+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:02:44.990212+0000","last_undegraded":"2024-09-22T01:02:44.990212+0000","last_fullsized":"2024-09-22T01:02:44.990212+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:55.260294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010459969999999999,"stat_sum":{"num_bytes":36976643,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":19,"num_read":1621,"num_read_kb":15577,"num_write":344,"num_write_kb":17862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":171,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.796841+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:02:45.796841+0000","last_peered":"2024-09-22T01:02:45.796841+0000","last_clean":"2024-09-22T01:02:45.796841+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:02:45.796841+0000","last_undegraded":"2024-09-22T01:02:45.796841+0000","last_fullsized":"2024-09-22T01:02:45.796841+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":367,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.959934+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:02:44.959934+0000","last_peered":"2024-09-22T01:02:44.959934+0000","last_clean":"2024-09-22T01:02:44.959934+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:02:44.959934+0000","last_undegraded":"2024-09-22T01:02:44.959934+0000","last_fullsized":"2024-09-22T01:02:44.959934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"119'360","reported_seq":10029,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990192+0000","last_change":"2024-09-22T01:02:39.736794+0000","last_active":"2024-09-22T01:02:44.990192+0000","last_peered":"2024-09-22T01:02:44.990192+0000","last_clean":"2024-09-22T01:02:44.990192+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:02:44.990192+0000","last_undegraded":"2024-09-22T01:02:44.990192+0000","last_fullsized":"2024-09-22T01:02:44.990192+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:53:34.004734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078129100000000004,"stat_sum":{"num_bytes":60097571,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":28,"num_read":10369,"num_read_kb":27110,"num_write":380,"num_write_kb":13691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"119'491","reported_seq":1921,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.038837+0000","last_change":"2024-09-22T01:01:43.142999+0000","last_active":"2024-09-22T01:02:45.038837+0000","last_peered":"2024-09-22T01:02:45.038837+0000","last_clean":"2024-09-22T01:02:45.038837+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:02:45.038837+0000","last_undegraded":"2024-09-22T01:02:45.038837+0000","last_fullsized":"2024-09-22T01:02:45.038837+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:16:43.137041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052701500000000004,"stat_sum":{"num_bytes":41881935,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":21,"num_read":2593,"num_read_kb":6970,"num_write":588,"num_write_kb":8014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"119'633","reported_seq":15168,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.812186+0000","last_change":"2024-09-22T01:02:45.812676+0000","last_active":"2024-09-22T01:02:45.812186+0000","last_peered":"2024-09-22T01:02:45.812186+0000","last_clean":"2024-09-22T01:02:45.812186+0000","last_became_active":"2024-09-22T01:02:37.737218+0000","last_became_peered":"2024-09-22T01:02:37.737218+0000","last_unstale":"2024-09-22T01:02:45.812186+0000","last_undegraded":"2024-09-22T01:02:45.812186+0000","last_fullsized":"2024-09-22T01:02:45.812186+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:48:25.554729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001187434,"stat_sum":{"num_bytes":62538447,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":21,"num_read":15928,"num_read_kb":39964,"num_write":697,"num_write_kb":13015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":13401696,"num_keys_recovered":167,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.b","version":"119'565","reported_seq":2256,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.039730+0000","last_change":"2024-09-22T01:00:49.736542+0000","last_active":"2024-09-22T01:02:45.039730+0000","last_peered":"2024-09-22T01:02:45.039730+0000","last_clean":"2024-09-22T01:02:45.039730+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:02:45.039730+0000","last_undegraded":"2024-09-22T01:02:45.039730+0000","last_fullsized":"2024-09-22T01:02:45.039730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:11.769709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183078,"stat_sum":{"num_bytes":22105578,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":3955,"num_read_kb":19562,"num_write":725,"num_write_kb":10845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"120'348","reported_seq":1033,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.911998+0000","last_change":"2024-09-22T01:02:45.798430+0000","last_active":"2024-09-22T01:02:45.911998+0000","last_peered":"2024-09-22T01:02:45.911998+0000","last_clean":"2024-09-22T01:02:45.911998+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:02:45.911998+0000","last_undegraded":"2024-09-22T01:02:45.911998+0000","last_fullsized":"2024-09-22T01:02:45.911998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:26:27.741662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012798309999999999,"stat_sum":{"num_bytes":48284657,"num_objects":54,"num_object_clones":18,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":2557,"num_read_kb":21657,"num_write":371,"num_write_kb":11071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.d","version":"119'633","reported_seq":5343,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960715+0000","last_change":"2024-09-22T01:01:27.940019+0000","last_active":"2024-09-22T01:02:44.960715+0000","last_peered":"2024-09-22T01:02:44.960715+0000","last_clean":"2024-09-22T01:02:44.960715+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:02:44.960715+0000","last_undegraded":"2024-09-22T01:02:44.960715+0000","last_fullsized":"2024-09-22T01:02:44.960715+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:49.738379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048777800000000002,"stat_sum":{"num_bytes":39623057,"num_objects":65,"num_object_clones":21,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":6994,"num_read_kb":28695,"num_write":813,"num_write_kb":17606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"118'400","reported_seq":4468,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960423+0000","last_change":"2024-09-22T01:02:38.741881+0000","last_active":"2024-09-22T01:02:44.960423+0000","last_peered":"2024-09-22T01:02:44.960423+0000","last_clean":"2024-09-22T01:02:44.960423+0000","last_became_active":"2024-09-22T01:02:38.737922+0000","last_became_peered":"2024-09-22T01:02:38.737922+0000","last_unstale":"2024-09-22T01:02:44.960423+0000","last_undegraded":"2024-09-22T01:02:44.960423+0000","last_fullsized":"2024-09-22T01:02:44.960423+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:52.350950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072492300000000004,"stat_sum":{"num_bytes":50428773,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":3758,"num_read_kb":26217,"num_write":469,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":14730069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"118'304","reported_seq":4337,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.960179+0000","last_change":"2024-09-22T01:01:43.142453+0000","last_active":"2024-09-22T01:02:44.960179+0000","last_peered":"2024-09-22T01:02:44.960179+0000","last_clean":"2024-09-22T01:02:44.960179+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:02:44.960179+0000","last_undegraded":"2024-09-22T01:02:44.960179+0000","last_fullsized":"2024-09-22T01:02:44.960179+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:48.324524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011434419999999999,"stat_sum":{"num_bytes":30409407,"num_objects":58,"num_object_clones":21,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":26,"num_read":4830,"num_read_kb":10051,"num_write":269,"num_write_kb":12202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"120'370","reported_seq":2840,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.915345+0000","last_change":"2024-09-22T01:02:45.811839+0000","last_active":"2024-09-22T01:02:45.915345+0000","last_peered":"2024-09-22T01:02:45.915345+0000","last_clean":"2024-09-22T01:02:45.915345+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:02:45.915345+0000","last_undegraded":"2024-09-22T01:02:45.915345+0000","last_fullsized":"2024-09-22T01:02:45.915345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:01:02.962107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061419399999999998,"stat_sum":{"num_bytes":44698178,"num_objects":69,"num_object_clones":22,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":31,"num_read":2996,"num_read_kb":14513,"num_write":349,"num_write_kb":12727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.11","version":"119'560","reported_seq":2332,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:45.083269+0000","last_change":"2024-09-22T01:01:43.139843+0000","last_active":"2024-09-22T01:02:45.083269+0000","last_peered":"2024-09-22T01:02:45.083269+0000","last_clean":"2024-09-22T01:02:45.083269+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:02:45.083269+0000","last_undegraded":"2024-09-22T01:02:45.083269+0000","last_fullsized":"2024-09-22T01:02:45.083269+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:27:34.182375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000743857,"stat_sum":{"num_bytes":33886177,"num_objects":52,"num_object_clones":19,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":3997,"num_read_kb":11283,"num_write":731,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"118'361","reported_seq":2694,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.961723+0000","last_change":"2024-09-22T01:00:49.729700+0000","last_active":"2024-09-22T01:02:44.961723+0000","last_peered":"2024-09-22T01:02:44.961723+0000","last_clean":"2024-09-22T01:02:44.961723+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:02:44.961723+0000","last_undegraded":"2024-09-22T01:02:44.961723+0000","last_fullsized":"2024-09-22T01:02:44.961723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:54.959321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082266299999999995,"stat_sum":{"num_bytes":26774974,"num_objects":59,"num_object_clones":21,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":28,"num_read":3252,"num_read_kb":7271,"num_write":366,"num_write_kb":8136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"119'274","reported_seq":6264,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T01:02:44.990312+0000","last_change":"2024-09-22T01:02:06.801024+0000","last_active":"2024-09-22T01:02:44.990312+0000","last_peered":"2024-09-22T01:02:44.990312+0000","last_clean":"2024-09-22T01:02:44.990312+0000","last_became_active":"2024-09-22T01:02:06.799165+0000","last_became_peered":"2024-09-22T01:02:06.799165+0000","last_unstale":"2024-09-22T01:02:44.990312+0000","last_undegraded":"2024-09-22T01:02:44.990312+0000","last_fullsized":"2024-09-22T01:02:44.990312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:02:44.962116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607259999999999,"stat_sum":{"num_bytes":30449431,"num_objects":47,"num_object_clones":16,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5691,"num_read_kb":26869,"num_write":330,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":11755308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1538224018,"num_objects":1912,"num_object_clones":586,"num_object_copies":3824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1912,"num_whiteouts":744,"num_read":190858,"num_read_kb":749856,"num_write":20430,"num_write_kb":410528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2782,"num_bytes_recovered":175106004,"num_keys_recovered":760,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":554139648,"data_stored":657590452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":16373,"ondisk_log_size":16373,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86352,"kb_used_data":59320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94285488,"statfs":{"total":96636764160,"available":96548339712,"internally_reserved":0,"allocated":60743680,"data_stored":70775326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299409,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132912,"kb_used_data":92312,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94238928,"statfs":{"total":96636764160,"available":96500662272,"internally_reserved":0,"allocated":94527488,"data_stored":120947515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":41545294},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130196,"kb_used_data":76800,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94241644,"statfs":{"total":96636764160,"available":96503443456,"internally_reserved":0,"allocated":78643200,"data_stored":90318886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542175,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150040,"kb_used_data":92476,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94221800,"statfs":{"total":96636764160,"available":96483123200,"internally_reserved":0,"allocated":94695424,"data_stored":111271363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74332,"kb_used_data":43136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297508,"statfs":{"total":96636764160,"available":96560648192,"internally_reserved":0,"allocated":44171264,"data_stored":53338065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133236,"kb_used_data":75672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94238604,"statfs":{"total":96636764160,"available":96500330496,"internally_reserved":0,"allocated":77488128,"data_stored":88854456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145264,"kb_used_data":87704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94226576,"statfs":{"total":96636764160,"available":96488013824,"internally_reserved":0,"allocated":89808896,"data_stored":98581825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87801856,"data_stored":97327426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":76070912,"data_stored":88190406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42758144,"data_stored":52674034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":93282304,"data_stored":110607332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":91549696,"data_stored":118742072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":59318272,"data_stored":70105063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":76566528,"data_stored":89008122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:02:47.360 INFO:tasks.thrashosds.thrasher:Setting 3.17 to [6, 1] 2024-09-22T01:02:47.361 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.17', '6', '1'] 2024-09-22T01:02:47.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.17 6 1 2024-09-22T01:02:47.374 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:47.374 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:47.442 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:47.437+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:47.472 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:47.472 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:47.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:47.472 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:47.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389653653, 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:02:47.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:02:47.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.38963546799999, 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:02:47.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.38961768, 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.38959912200001, 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:02:47.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.38958107000001, 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:02:47.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389563721, 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:02:47.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389541768, 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389524233, 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:02:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389500447, 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:02:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:02:47.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.519 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389482585, 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:02:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389455348, 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:47.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:47.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.530 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389436643, 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.531 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:02:47.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.535 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389413146, 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:02:47.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.541 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389389177, 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:02:47.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:02:47.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.546 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.389369962, 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:02:47.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:47.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.388365412, 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:02:47.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.557 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 112.385859459, 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:02:47.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:47.563 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.670401299000002, 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:02:47.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:02:47.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:140499 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:47.570 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.411725129000001, 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98072477000000002, 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140499 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057286+0000", 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057297+0000", 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026e-05 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057320+0000", 2024-09-22T01:02:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3391999999999999e-05 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057331+0000", 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.075e-05 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057403+0000", 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057411+0000", 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2749999999999995e-06 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.037974+0000", 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98056295800000004 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.038008+0000", 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3844999999999999e-05 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:144964 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.6874695699999993, 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95820099800000003, 2024-09-22T01:02:47.576 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144964 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:47.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781542+0000", 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4899999999999999e-06 2024-09-22T01:02:47.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781553+0000", 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1687e-05 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781583+0000", 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0173999999999999e-05 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781597+0000", 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3172e-05 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781681+0000", 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4810999999999998e-05 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:47.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781692+0000", 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0662000000000001e-05 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739463+0000", 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95777067000000005 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739603+0000", 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013200000000001 2024-09-22T01:02:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739740+0000", 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001372 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:47.582 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:47.584 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:47.585 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:47.581+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:47.685 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:47.681+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:47.710 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:47.711 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:47.711 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:47.711 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:47.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:47.786 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:47.785+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:47.813 INFO:teuthology.orchestra.run.smithi002.stderr:set 3.17 pg_upmap mapping to [6,1] 2024-09-22T01:02:47.869 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:47.869 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:47.870 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:47.870 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:47.870 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:47.887 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:47.887 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:47.885+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:47.988 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:47.981+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:48.064 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:48.064 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3.262a7b13 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 44.290111093, 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366911223, 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771094+0000", 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0930000000000001e-06 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771103+0000", 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0629999999999995e-06 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771127+0000", 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3924e-05 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771147+0000", 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:02:48.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771214+0000", 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7348999999999995e-05 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771220+0000", 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3709999999999996e-06 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.137980+0000", 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366760011 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138003+0000", 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2823000000000002e-05 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3.c7ddc9b3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 44.290023347999998, 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.367176741, 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:48.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:48.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771181+0000", 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771231+0000", 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-22T01:02:48.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771247+0000", 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5858999999999998e-05 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771259+0000", 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2814e-05 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771320+0000", 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0393999999999998e-05 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771325+0000", 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138340+0000", 2024-09-22T01:02:48.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3670150409999999 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138356+0000", 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6782999999999998e-05 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3:c8de5464:::rbd_data.117c2270b223.000000000000001e:head [sparse-read 0~4194304] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 44.289101649999999, 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366676402, 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.077 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772103+0000", 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9929999999999998e-06 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772113+0000", 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.499e-06 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772138+0000", 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4743999999999999e-05 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772151+0000", 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3189e-05 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772415+0000", 2024-09-22T01:02:48.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026382499999999998 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772427+0000", 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2103000000000001e-05 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138485+0000", 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366057984 2024-09-22T01:02:48.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138778+0000", 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029306500000000001 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3:cd93bbe3:::rbd_data.117c2270b223.0000000000000038:head [sparse-read 0~3145728] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:48.082 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 44.289025535, 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366907898, 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5600000000000005e-07 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772183+0000", 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772195+0000", 2024-09-22T01:02:48.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1533e-05 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772206+0000", 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1647e-05 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772493+0000", 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028702699999999997 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772511+0000", 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7609999999999999e-05 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138879+0000", 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3663676759999999 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.139085+0000", 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020690899999999999 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:90676 3.9 3:963e58c3:::rbd_data.117c77dccaa7.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.519093882999996, 2024-09-22T01:02:48.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093363432999999996, 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 90676 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542112+0000", 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9730000000000002e-06 2024-09-22T01:02:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542126+0000", 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542161+0000", 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4975000000000003e-05 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542185+0000", 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4613999999999998e-05 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542321+0000", 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013617999999999999 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542562+0000", 2024-09-22T01:02:48.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024014899999999999 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.556142+0000", 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013580126 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635270+0000", 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079128575000000007 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635299+0000", 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8996000000000002e-05 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635391+0000", 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1712000000000006e-05 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635473+0000", 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1588999999999994e-05 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104388 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [write 9699485~40067] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.128471837999999, 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896004440000002, 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104388 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932733+0000", 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1780000000000002e-06 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932760+0000", 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6174e-05 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932775+0000", 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5603e-05 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932787+0000", 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2155e-05 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932873+0000", 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5391999999999998e-05 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932887+0000", 2024-09-22T01:02:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3915e-05 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930412+0000", 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.99752547400000002 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930446+0000", 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3841999999999999e-05 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.019619+0000", 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.089172569 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020035+0000", 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041613899999999999 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-22T01:02:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020214+0000", 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017929999999999999 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.021438+0000", 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012240440000000001 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022156+0000", 2024-09-22T01:02:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071828699999999996 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022231+0000", 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420999999999998e-05 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022332+0000", 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100951 2024-09-22T01:02:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106194 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 778240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.677629241999998, 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091629434999999995, 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106194 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383577+0000", 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7549999999999999e-06 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383585+0000", 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4209999999999998e-06 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383599+0000", 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4599e-05 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.474958+0000", 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091358710999999995 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475039+0000", 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1075999999999994e-05 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475203+0000", 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016427600000000001 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106195 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.677594399, 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091906246999999996, 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106195 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383612+0000", 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383620+0000", 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0600000000000008e-06 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383626+0000", 2024-09-22T01:02:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8529999999999997e-06 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475245+0000", 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091618623999999996 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475326+0000", 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1166000000000002e-05 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475515+0000", 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018890399999999999 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106196 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:48.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.677568376, 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092159059000000002, 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106196 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383645+0000", 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7119999999999995e-06 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383653+0000", 2024-09-22T01:02:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.807e-06 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383658+0000", 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.04e-06 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475547+0000", 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091889557999999996 2024-09-22T01:02:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475630+0000", 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3114000000000004e-05 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475794+0000", 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016325199999999999 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106197 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 821248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.677535676000002, 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092422578000000005, 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106197 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383668+0000", 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.63e-07 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383677+0000", 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8859999999999993e-06 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383685+0000", 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5609999999999999e-06 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383690+0000", 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:02:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475825+0000", 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092134439999999998 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475917+0000", 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2559000000000006e-05 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476090+0000", 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017251899999999999 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106198 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.677502515, 2024-09-22T01:02:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092626416000000003, 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106198 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384316+0000", 2024-09-22T01:02:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061580100000000002 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384323+0000", 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384338+0000", 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5126999999999999e-05 2024-09-22T01:02:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384348+0000", 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8479999999999996e-06 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476109+0000", 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091760280999999999 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476178+0000", 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9865999999999998e-05 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476327+0000", 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014848199999999999 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106199 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.676838098000001, 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092274072999999998, 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106199 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385301+0000", 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093565 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385312+0000", 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1589e-05 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385342+0000", 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9743999999999998e-05 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385361+0000", 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9219999999999999e-05 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476358+0000", 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090996442999999996 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476438+0000", 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9977999999999999e-05 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476639+0000", 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000201449 2024-09-22T01:02:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106200 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.675810111000001, 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091589137000000001, 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106200 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385394+0000", 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1599999999999999e-06 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385401+0000", 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9090000000000003e-06 2024-09-22T01:02:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385420+0000", 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385434+0000", 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4068e-05 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476677+0000", 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091242904999999999 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476760+0000", 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3783999999999996e-05 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476982+0000", 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022163200000000001 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106201 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.675588626, 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091715669, 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106201 2024-09-22T01:02:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385616+0000", 2024-09-22T01:02:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.277e-06 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385624+0000", 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7910000000000005e-06 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385646+0000", 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2251000000000001e-05 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385680+0000", 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3933000000000002e-05 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477014+0000", 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091334175000000004 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477096+0000", 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2361999999999995e-05 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477330+0000", 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023388 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106202 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.675500208999999, 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091937313000000007, 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106202 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385835+0000", 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013252699999999999 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385844+0000", 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1599999999999998e-06 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385864+0000", 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0579000000000001e-05 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385879+0000", 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4871e-05 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477362+0000", 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091483156999999996 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477452+0000", 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9868000000000001e-05 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477640+0000", 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018815100000000001 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106203 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.675281376000001, 2024-09-22T01:02:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092020950000000004, 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106203 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386095+0000", 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017321200000000001 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386104+0000", 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5109999999999992e-06 2024-09-22T01:02:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386129+0000", 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4785000000000001e-05 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386145+0000", 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6191000000000001e-05 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477675+0000", 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091529570000000005 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477762+0000", 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6861999999999996e-05 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477943+0000", 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180819 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106204 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.675030937999999, 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092064394999999993, 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106204 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386406+0000", 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023343699999999999 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386412+0000", 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386425+0000", 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2719e-05 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386433+0000", 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2810000000000008e-06 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477976+0000", 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091542739999999997 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478069+0000", 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2971000000000001e-05 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478237+0000", 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167949 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106205 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.674755270999999, 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092018127000000005, 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106205 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387017+0000", 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056957000000000004 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387024+0000", 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387035+0000", 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1187e-05 2024-09-22T01:02:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387045+0000", 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0750000000000004e-06 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478259+0000", 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091214394000000004 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478311+0000", 2024-09-22T01:02:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1984000000000002e-05 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478466+0000", 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015504200000000001 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106206 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.674139884999999, 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091679640000000007, 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106206 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387533+0000", 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047020100000000001 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387539+0000", 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8200000000000002e-06 2024-09-22T01:02:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387552+0000", 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2865e-05 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387560+0000", 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5499999999999997e-06 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478507+0000", 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090947741999999998 2024-09-22T01:02:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478599+0000", 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1181000000000007e-05 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478743+0000", 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014428100000000001 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106208 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.673591082000002, 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091664238999999995, 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106208 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391033+0000", 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00342144 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391040+0000", 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391054+0000", 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4089e-05 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391063+0000", 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3809999999999999e-06 2024-09-22T01:02:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479014+0000", 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087951647999999993 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479099+0000", 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4859999999999997e-05 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479276+0000", 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017700699999999999 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.171 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:48.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:48.173 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:48.169+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:48.273 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:48.269+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:48.347 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:48.347 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:48.347 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:48.347 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:48.363 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:48.406 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:48.405+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:48.475 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:48.473+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:48.528 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:48.528 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:48.542 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:48.576 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:48.569+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:48.677 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:48.673+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234394992000006, 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.715 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234374162999998, 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:48.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:48.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234345648000001, 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:48.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234322997999996, 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234301002999999, 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.728 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:48.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:48.732 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234275939, 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:48.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234253506000002, 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:48.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234232028999998, 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:48.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:48.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.234203270999998, 2024-09-22T01:02:48.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:48.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:48.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.233390279000005, 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:48.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:48.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.233365594000006, 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:48.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.229044060000007, 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:48.758 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:48.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.229015208999996, 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:48.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:48.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.228993484, 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.771 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.228973053999994, 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:48.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:48.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.228950737999995, 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.776 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:48.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.228882943000002, 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:48.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.228849805999999, 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:48.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:48.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.228825884000003, 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:48.793 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.228802286999993, 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:48.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:48.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:48.798 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:48.798 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:48.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:48.799 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:48.797+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:48.843 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:02:48.900 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:48.897+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:48.962 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:48.962 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:48.962 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:48.962 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:48.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:49.025 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:49.021+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:49.101 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:49.102 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:49.102 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:49.097+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:49.115 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:49.202 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:49.197+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:49.285 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:49.285 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:49.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:49.285 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:49.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.285 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313699518999996, 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:02:49.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313646996000003, 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.290 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:02:49.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313594675000004, 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:02:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313428900999995, 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:02:49.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313400473000002, 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.302 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:02:49.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313372663999999, 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:02:49.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313344559000001, 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:02:49.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313317017000003, 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:02:49.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313289369000003, 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313262223999999, 2024-09-22T01:02:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:02:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:02:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313234265000006, 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313206449999996, 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:02:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313179504999994, 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313151804, 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.313037379999997, 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.312998022000002, 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.312967866999998, 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.312938609, 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.312910732999995, 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.312881359000002, 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:02:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:02:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:49.365 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:49.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:49.367 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:49.361+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:49.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:49.466 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:49.461+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:49.537 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:49.537 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:49.537 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:49.537 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:49.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:49.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:49.554 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:49.568 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:49.565+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:49.571 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:49.571 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:49.587 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:49.588 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:49.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:49.604 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:49.623 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:49.623 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:49.643 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:49.643 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:49.664 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:49.665 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:49.669 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:49.665+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:49.687 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:49.687 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:49.721 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:49.721 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:49.737 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:49.770 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:49.769+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:49.826 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:02:49.861 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:49.933 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:49.935 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:49.933+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:49.935 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:50.035 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:50.029+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:50.069 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:50.069 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:50.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:50.135 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:50.133+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:50.236 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:50.233+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:50.263 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:50.263 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:50.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:50.336 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:50.329+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:50.399 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:50.399 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:50.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:50.437 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:50.433+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:50.539 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:50.537+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:50.586 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:50.586 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:50.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:50.639 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:50.637+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:50.717 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:50.717 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:50.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:50.740 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:50.737+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:50.841 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:50.837+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:50.855 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:50.855 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:50.875 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:50.942 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:50.937+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:51.025 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:51.025 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:51.042 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:51.043 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:51.041+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:51.143 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:51.141+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:51.214 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:51.214 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:51.228 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:51.270 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:51.269+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:51.346 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:51.341+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:51.397 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:51.398 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:51.413 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:51.458 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:51.457+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:51.537 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:51.538 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:51.547 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:51.545+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:51.552 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:51.649 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:51.645+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:51.706 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:51.715 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:51.715 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:51.730 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:51.749 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:51.749+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:51.850 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:51.845+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:51.878 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:51.879 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:51.901 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:51.901 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835772283, 2024-09-22T01:02:51.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:02:51.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:02:51.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835754098, 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:02:51.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.83573631, 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:51.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:02:51.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.83571775199999, 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:02:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.83569970000001, 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:02:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:02:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:02:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835682351, 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:02:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835660398, 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:02:51.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835642863, 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:02:51.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:02:51.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835619077, 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:02:51.952 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:02:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:02:51.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:02:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.956 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835601215, 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:51.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835573978, 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:02:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:02:51.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835555273, 2024-09-22T01:02:51.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:51.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:02:51.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:02:51.969 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835531776, 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.970 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835507807, 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:02:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.835488592, 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.979 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:02:51.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:02:51.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.982 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.834484042, 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:02:51.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.831978089, 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:02:51.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:02:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.116519928999999, 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:51.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:02:51.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:51.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:02:51.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:140499 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.857843759000001, 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98072477000000002, 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140499 2024-09-22T01:02:51.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057286+0000", 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-22T01:02:51.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057297+0000", 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026e-05 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057320+0000", 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3391999999999999e-05 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057331+0000", 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.075e-05 2024-09-22T01:02:51.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057403+0000", 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057411+0000", 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2749999999999995e-06 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.037974+0000", 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98056295800000004 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.038008+0000", 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3844999999999999e-05 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:144964 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.1335882, 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95820099800000003, 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144964 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781542+0000", 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4899999999999999e-06 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781553+0000", 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1687e-05 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781583+0000", 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0173999999999999e-05 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781597+0000", 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3172e-05 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781681+0000", 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4810999999999998e-05 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781692+0000", 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0662000000000001e-05 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739463+0000", 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95777067000000005 2024-09-22T01:02:52.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739603+0000", 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013200000000001 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739740+0000", 2024-09-22T01:02:52.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001372 2024-09-22T01:02:52.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.008 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:52.008 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:02:52.010 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:52.011 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:52.012 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:52.013 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:52.013 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:52.014 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:52.015 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:52.009+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:52.032 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:52.033 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:52.111 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:52.109+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:52.153 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:52.153 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:52.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:52.212 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:52.209+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:52.286 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:52.286 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:52.286 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:52.286 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:52.286 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:52.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:52.346 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:52.345+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:52.413 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:52.409+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:52.423 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:52.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:52.423 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:52.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.423 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3.262a7b13 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.649235689000001, 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366911223, 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771094+0000", 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0930000000000001e-06 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771103+0000", 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0629999999999995e-06 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771127+0000", 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3924e-05 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771147+0000", 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771214+0000", 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7348999999999995e-05 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771220+0000", 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3709999999999996e-06 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.137980+0000", 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366760011 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138003+0000", 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2823000000000002e-05 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3.c7ddc9b3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.649147943999999, 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.367176741, 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:52.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771181+0000", 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771231+0000", 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771247+0000", 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5858999999999998e-05 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771259+0000", 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2814e-05 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771320+0000", 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0393999999999998e-05 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771325+0000", 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138340+0000", 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3670150409999999 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138356+0000", 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6782999999999998e-05 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.432 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3:c8de5464:::rbd_data.117c2270b223.000000000000001e:head [sparse-read 0~4194304] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.648226246, 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366676402, 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.433 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772103+0000", 2024-09-22T01:02:52.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9929999999999998e-06 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772113+0000", 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.499e-06 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772138+0000", 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4743999999999999e-05 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772151+0000", 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3189e-05 2024-09-22T01:02:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772415+0000", 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026382499999999998 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772427+0000", 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2103000000000001e-05 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138485+0000", 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366057984 2024-09-22T01:02:52.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138778+0000", 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029306500000000001 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3:cd93bbe3:::rbd_data.117c2270b223.0000000000000038:head [sparse-read 0~3145728] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.648150131000001, 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366907898, 2024-09-22T01:02:52.437 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5600000000000005e-07 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772183+0000", 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772195+0000", 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1533e-05 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772206+0000", 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1647e-05 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772493+0000", 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028702699999999997 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772511+0000", 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7609999999999999e-05 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138879+0000", 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3663676759999999 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.139085+0000", 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020690899999999999 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:90676 3.9 3:963e58c3:::rbd_data.117c77dccaa7.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.878218478999997, 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093363432999999996, 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 90676 2024-09-22T01:02:52.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542112+0000", 2024-09-22T01:02:52.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9730000000000002e-06 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542126+0000", 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542161+0000", 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4975000000000003e-05 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542185+0000", 2024-09-22T01:02:52.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4613999999999998e-05 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542321+0000", 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013617999999999999 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542562+0000", 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024014899999999999 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.556142+0000", 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013580126 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635270+0000", 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079128575000000007 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635299+0000", 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8996000000000002e-05 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635391+0000", 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1712000000000006e-05 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635473+0000", 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1588999999999994e-05 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104388 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [write 9699485~40067] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.487596434, 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896004440000002, 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:52.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104388 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932733+0000", 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1780000000000002e-06 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932760+0000", 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6174e-05 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932775+0000", 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5603e-05 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932787+0000", 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2155e-05 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932873+0000", 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5391999999999998e-05 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932887+0000", 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3915e-05 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930412+0000", 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.99752547400000002 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930446+0000", 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3841999999999999e-05 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.019619+0000", 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.089172569 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020035+0000", 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041613899999999999 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020214+0000", 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017929999999999999 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.021438+0000", 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012240440000000001 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022156+0000", 2024-09-22T01:02:52.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071828699999999996 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022231+0000", 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420999999999998e-05 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022332+0000", 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100951 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106194 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 778240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.036753837999999, 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091629434999999995, 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106194 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383577+0000", 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7549999999999999e-06 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383585+0000", 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4209999999999998e-06 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383599+0000", 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4599e-05 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.474958+0000", 2024-09-22T01:02:52.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091358710999999995 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475039+0000", 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1075999999999994e-05 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475203+0000", 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016427600000000001 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106195 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.036718995000001, 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091906246999999996, 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106195 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.458 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:02:52.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383612+0000", 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383620+0000", 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0600000000000008e-06 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383626+0000", 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8529999999999997e-06 2024-09-22T01:02:52.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475245+0000", 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091618623999999996 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475326+0000", 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1166000000000002e-05 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475515+0000", 2024-09-22T01:02:52.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018890399999999999 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106196 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.036692972000001, 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092159059000000002, 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106196 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383645+0000", 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7119999999999995e-06 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383653+0000", 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.807e-06 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383658+0000", 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.04e-06 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475547+0000", 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091889557999999996 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475630+0000", 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3114000000000004e-05 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475794+0000", 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016325199999999999 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106197 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 821248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.036660271999999, 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092422578000000005, 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106197 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383668+0000", 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.63e-07 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383677+0000", 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8859999999999993e-06 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383685+0000", 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5609999999999999e-06 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383690+0000", 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475825+0000", 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092134439999999998 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475917+0000", 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2559000000000006e-05 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476090+0000", 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017251899999999999 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106198 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.036627111000001, 2024-09-22T01:02:52.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092626416000000003, 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106198 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384316+0000", 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061580100000000002 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384323+0000", 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384338+0000", 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5126999999999999e-05 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384348+0000", 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8479999999999996e-06 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476109+0000", 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091760280999999999 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476178+0000", 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9865999999999998e-05 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476327+0000", 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014848199999999999 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106199 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.474 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.035962693999998, 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092274072999999998, 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106199 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385301+0000", 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093565 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385312+0000", 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1589e-05 2024-09-22T01:02:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385342+0000", 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9743999999999998e-05 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385361+0000", 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9219999999999999e-05 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476358+0000", 2024-09-22T01:02:52.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090996442999999996 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476438+0000", 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9977999999999999e-05 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476639+0000", 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000201449 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106200 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.034934707000001, 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091589137000000001, 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106200 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385394+0000", 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1599999999999999e-06 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385401+0000", 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9090000000000003e-06 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385420+0000", 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385434+0000", 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4068e-05 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476677+0000", 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091242904999999999 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476760+0000", 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3783999999999996e-05 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476982+0000", 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022163200000000001 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106201 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.034713222000001, 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091715669, 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106201 2024-09-22T01:02:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385616+0000", 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.277e-06 2024-09-22T01:02:52.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385624+0000", 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7910000000000005e-06 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385646+0000", 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2251000000000001e-05 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385680+0000", 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3933000000000002e-05 2024-09-22T01:02:52.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477014+0000", 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091334175000000004 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477096+0000", 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2361999999999995e-05 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477330+0000", 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023388 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.486 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106202 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.034624805, 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091937313000000007, 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106202 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385835+0000", 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013252699999999999 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385844+0000", 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1599999999999998e-06 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385864+0000", 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0579000000000001e-05 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385879+0000", 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4871e-05 2024-09-22T01:02:52.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477362+0000", 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091483156999999996 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477452+0000", 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9868000000000001e-05 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477640+0000", 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018815100000000001 2024-09-22T01:02:52.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106203 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.034405971999998, 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092020950000000004, 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106203 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386095+0000", 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017321200000000001 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386104+0000", 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5109999999999992e-06 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386129+0000", 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4785000000000001e-05 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386145+0000", 2024-09-22T01:02:52.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6191000000000001e-05 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477675+0000", 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091529570000000005 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477762+0000", 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6861999999999996e-05 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477943+0000", 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180819 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106204 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.034155534, 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092064394999999993, 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.495 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106204 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386406+0000", 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023343699999999999 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386412+0000", 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386425+0000", 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2719e-05 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386433+0000", 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2810000000000008e-06 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477976+0000", 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091542739999999997 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478069+0000", 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2971000000000001e-05 2024-09-22T01:02:52.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478237+0000", 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167949 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106205 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.033879867, 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092018127000000005, 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106205 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387017+0000", 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056957000000000004 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387024+0000", 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387035+0000", 2024-09-22T01:02:52.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1187e-05 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387045+0000", 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0750000000000004e-06 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478259+0000", 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091214394000000004 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478311+0000", 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1984000000000002e-05 2024-09-22T01:02:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478466+0000", 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015504200000000001 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106206 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.033264481, 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091679640000000007, 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106206 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387533+0000", 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047020100000000001 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387539+0000", 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8200000000000002e-06 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387552+0000", 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2865e-05 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387560+0000", 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5499999999999997e-06 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478507+0000", 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090947741999999998 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478599+0000", 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1181000000000007e-05 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478743+0000", 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014428100000000001 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106208 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:52.507 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.032715677999999, 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091664238999999995, 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106208 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:52.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391033+0000", 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00342144 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391040+0000", 2024-09-22T01:02:52.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391054+0000", 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4089e-05 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391063+0000", 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3809999999999999e-06 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479014+0000", 2024-09-22T01:02:52.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087951647999999993 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479099+0000", 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4859999999999997e-05 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479276+0000", 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017700699999999999 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.512 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:52.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:52.514 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:52.513+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:52.615 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:52.613+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:52.684 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:52.684 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:52.700 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:52.715 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:52.713+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:52.816 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:52.813+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:52.828 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:52.828 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:52.828 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:52.828 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:52.828 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:52.842 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:52.843 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:02:52.843 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:02:52.843 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-22T01:02:52.843 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:02:52.843 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:02:52.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-22T01:02:52.868 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:02:52.917 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:52.913+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:52.993 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:52.993 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "" 2024-09-22T01:02:52.993 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:52.994 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519241369, 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519220540000006, 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519192024999995, 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519169375000004, 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519147380000007, 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519122315999994, 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519099882999996, 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519078406000006, 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.519049648000006, 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.518236655999999, 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.518211970999999, 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513890437000001, 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:53.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513861586000004, 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:53.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:53.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:53.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513839860999994, 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:53.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:53.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513819431000002, 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:53.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513797115000003, 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513729319999996, 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.067 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513696182999993, 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:53.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513672260999996, 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:53.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:53.076 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.513648664000002, 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:53.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:53.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:53.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:53.080 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:53.082 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:53.083 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:53.081+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:53.183 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:53.181+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:53.254 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:53.255 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:53.255 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:53.255 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:53.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:53.283 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:53.281+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:53.384 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:53.377+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:53.391 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:53.391 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:53.391 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:53.391 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:53.391 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:53.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:53.485 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:53.481+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:53.579 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607960997000006, 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:02:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607908473999998, 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:02:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:02:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607856153, 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607690379000005, 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:02:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:02:53.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607661950999997, 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:02:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:02:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607634141999995, 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:02:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607606036999996, 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:02:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:02:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:53.607 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607578494999998, 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.611 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607550846999999, 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:02:53.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:53.615 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607523701999995, 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:53.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.619 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607495743000001, 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:02:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607467928000005, 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:02:53.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607440983000004, 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:53.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:02:53.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:02:53.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607413281999996, 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.631 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607298858000007, 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607259499999998, 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:02:53.639 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:02:53.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607229344999993, 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:02:53.643 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:02:53.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607200086999995, 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.647 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:02:53.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607172211000005, 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:02:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:02:53.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:02:53.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.607142836999998, 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:02:53.655 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:02:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:53.659 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:53.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:53.661 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:53.657+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:53.761 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:53.761+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:53.829 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:53.829 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:53.829 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:53.829 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:53.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:53.862 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:53.861+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:53.964 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:53.961+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:54.011 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:54.011 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:54.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:54.055 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:54.069 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:54.069+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:54.166 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:54.165+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:54.199 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:54.199 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:54.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:54.199 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:54.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840500734000003, 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:02:54.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840463765000003, 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.204 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840435495999998, 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:02:54.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:02:54.211 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840397619999997, 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840359030000002, 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:02:54.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840286702, 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:02:54.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:02:54.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840248836000001, 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840206596999998, 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.229 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:02:54.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:02:54.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840152212, 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.840051447, 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.237 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:54.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.241 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.839964522000002, 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:02:54.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:02:54.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.839906198000001, 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.839453048000003, 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:02:54.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:54.254 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.838814562000003, 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:02:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:02:54.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.838526965, 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.259 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:02:54.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.837923996000001, 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:02:54.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:02:54.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.836843696999999, 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:02:54.267 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.271 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.836773819000001, 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:02:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.836211828000003, 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:02:54.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.835834403, 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:54.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:54.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:02:54.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:54.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:54.284 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:54.285 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:54.286 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:54.287 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:54.287 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:54.287 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:54.288 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:54.288 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:54.281+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:54.290 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:54.290 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:54.311 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:54.333 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:54.333 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:54.353 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:54.353 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:54.386 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:54.385+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:54.456 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:54.456 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:54.470 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:54.487 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:54.485+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:54.588 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:54.585+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:54.635 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:54.636 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:54.650 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:54.693 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:54.693+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:54.766 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:54.766 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:54.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:54.790 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:54.789+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:54.892 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:54.889+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:54.897 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:02:54.951 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:54.951 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:54.967 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:55.017 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:55.017+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:55.093 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:55.093+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:55.103 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:55.103 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:55.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:55.209 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:55.209+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:55.286 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:55.286 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:55.294 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:55.293+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:55.300 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:55.395 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:55.393+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:55.428 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:55.428 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:55.442 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:55.496 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:55.493+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:55.561 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:55.562 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:55.576 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:55.618 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:55.617+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:55.698 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:55.697+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:55.739 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:55.739 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:55.753 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:02:55.802 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:55.801+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:02:55.900 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:55.897+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:55.942 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:55.942 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:55.957 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:02:56.042 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:56.041+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:56.102 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:56.101+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:56.132 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:56.132 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:56.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:02:56.238 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:56.237+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:56.304 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:56.301+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:56.314 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:56.314 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:56.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:02:56.372 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:02:56.404 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:56.401+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:56.498 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:56.498 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:56.505 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:56.501+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:56.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:02:56.557 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:56.557 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:56.574 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:56.574 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:56.592 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:56.592 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:56.606 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:56.605+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:56.609 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:56.609 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:56.630 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:56.630 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:56.650 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:56.650 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:56.671 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:56.671 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:56.674 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:56.674 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:56.689 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:02:56.693 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:56.693 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:56.707 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:56.705+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:56.808 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:56.805+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:56.813 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3.262a7b13 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.041303614, 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366911223, 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771094+0000", 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0930000000000001e-06 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771103+0000", 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0629999999999995e-06 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771127+0000", 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3924e-05 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771147+0000", 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771214+0000", 2024-09-22T01:02:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7348999999999995e-05 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771220+0000", 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3709999999999996e-06 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.137980+0000", 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366760011 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138003+0000", 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2823000000000002e-05 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3.c7ddc9b3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.041215868999998, 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.367176741, 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771181+0000", 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771231+0000", 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-22T01:02:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771247+0000", 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5858999999999998e-05 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771259+0000", 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2814e-05 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771320+0000", 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0393999999999998e-05 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771325+0000", 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138340+0000", 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3670150409999999 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138356+0000", 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6782999999999998e-05 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3:c8de5464:::rbd_data.117c2270b223.000000000000001e:head [sparse-read 0~4194304] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.040294170999999, 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366676402, 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772103+0000", 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9929999999999998e-06 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772113+0000", 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.499e-06 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772138+0000", 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4743999999999999e-05 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772151+0000", 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3189e-05 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772415+0000", 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026382499999999998 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772427+0000", 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2103000000000001e-05 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138485+0000", 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366057984 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138778+0000", 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029306500000000001 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3:cd93bbe3:::rbd_data.117c2270b223.0000000000000038:head [sparse-read 0~3145728] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.040218056000001, 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366907898, 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5600000000000005e-07 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772183+0000", 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772195+0000", 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1533e-05 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772206+0000", 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1647e-05 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772493+0000", 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028702699999999997 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772511+0000", 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7609999999999999e-05 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138879+0000", 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3663676759999999 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.139085+0000", 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020690899999999999 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:90676 3.9 3:963e58c3:::rbd_data.117c77dccaa7.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.270286403999997, 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093363432999999996, 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 90676 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542112+0000", 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9730000000000002e-06 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542126+0000", 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542161+0000", 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4975000000000003e-05 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542185+0000", 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4613999999999998e-05 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542321+0000", 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013617999999999999 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542562+0000", 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024014899999999999 2024-09-22T01:02:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.556142+0000", 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013580126 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635270+0000", 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079128575000000007 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635299+0000", 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8996000000000002e-05 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635391+0000", 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1712000000000006e-05 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635473+0000", 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1588999999999994e-05 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104388 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [write 9699485~40067] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.879664359000003, 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896004440000002, 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104388 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932733+0000", 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1780000000000002e-06 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932760+0000", 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6174e-05 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932775+0000", 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5603e-05 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932787+0000", 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2155e-05 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932873+0000", 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5391999999999998e-05 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932887+0000", 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3915e-05 2024-09-22T01:02:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930412+0000", 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.99752547400000002 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930446+0000", 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3841999999999999e-05 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.019619+0000", 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.089172569 2024-09-22T01:02:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020035+0000", 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041613899999999999 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020214+0000", 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017929999999999999 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.021438+0000", 2024-09-22T01:02:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012240440000000001 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022156+0000", 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071828699999999996 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022231+0000", 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420999999999998e-05 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022332+0000", 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100951 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106194 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 778240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.428821762999998, 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091629434999999995, 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106194 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383577+0000", 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7549999999999999e-06 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383585+0000", 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4209999999999998e-06 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383599+0000", 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4599e-05 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.474958+0000", 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091358710999999995 2024-09-22T01:02:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475039+0000", 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1075999999999994e-05 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475203+0000", 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016427600000000001 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106195 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.42878692, 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091906246999999996, 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106195 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:02:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383612+0000", 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383620+0000", 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0600000000000008e-06 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383626+0000", 2024-09-22T01:02:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8529999999999997e-06 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475245+0000", 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091618623999999996 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475326+0000", 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1166000000000002e-05 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475515+0000", 2024-09-22T01:02:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018890399999999999 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106196 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.428760897, 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092159059000000002, 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106196 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383645+0000", 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7119999999999995e-06 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383653+0000", 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.807e-06 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383658+0000", 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.04e-06 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475547+0000", 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091889557999999996 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475630+0000", 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3114000000000004e-05 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475794+0000", 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016325199999999999 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106197 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 821248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.428728197000002, 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092422578000000005, 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106197 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383668+0000", 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.63e-07 2024-09-22T01:02:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383677+0000", 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8859999999999993e-06 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383685+0000", 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5609999999999999e-06 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383690+0000", 2024-09-22T01:02:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475825+0000", 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092134439999999998 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475917+0000", 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2559000000000006e-05 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476090+0000", 2024-09-22T01:02:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017251899999999999 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106198 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.428695036000001, 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092626416000000003, 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106198 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384316+0000", 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061580100000000002 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384323+0000", 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384338+0000", 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5126999999999999e-05 2024-09-22T01:02:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384348+0000", 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8479999999999996e-06 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476109+0000", 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091760280999999999 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476178+0000", 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9865999999999998e-05 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476327+0000", 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014848199999999999 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106199 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.428030619000001, 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092274072999999998, 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106199 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385301+0000", 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093565 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385312+0000", 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1589e-05 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385342+0000", 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9743999999999998e-05 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385361+0000", 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9219999999999999e-05 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476358+0000", 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090996442999999996 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476438+0000", 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9977999999999999e-05 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476639+0000", 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000201449 2024-09-22T01:02:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106200 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.427002632000001, 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091589137000000001, 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106200 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385394+0000", 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1599999999999999e-06 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385401+0000", 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9090000000000003e-06 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385420+0000", 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:02:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385434+0000", 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4068e-05 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476677+0000", 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091242904999999999 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476760+0000", 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3783999999999996e-05 2024-09-22T01:02:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476982+0000", 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022163200000000001 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106201 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.426781147, 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091715669, 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106201 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385616+0000", 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.277e-06 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385624+0000", 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7910000000000005e-06 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385646+0000", 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2251000000000001e-05 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385680+0000", 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3933000000000002e-05 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477014+0000", 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091334175000000004 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477096+0000", 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2361999999999995e-05 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477330+0000", 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023388 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106202 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.426692729999999, 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091937313000000007, 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106202 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385835+0000", 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013252699999999999 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385844+0000", 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1599999999999998e-06 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385864+0000", 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0579000000000001e-05 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385879+0000", 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4871e-05 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477362+0000", 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091483156999999996 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477452+0000", 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9868000000000001e-05 2024-09-22T01:02:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477640+0000", 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018815100000000001 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106203 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.426473897000001, 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092020950000000004, 2024-09-22T01:02:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106203 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386095+0000", 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017321200000000001 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386104+0000", 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5109999999999992e-06 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386129+0000", 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4785000000000001e-05 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386145+0000", 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6191000000000001e-05 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477675+0000", 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091529570000000005 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477762+0000", 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6861999999999996e-05 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477943+0000", 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180819 2024-09-22T01:02:56.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106204 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.426223458999999, 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092064394999999993, 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.892 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106204 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:02:56.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386406+0000", 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023343699999999999 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386412+0000", 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386425+0000", 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2719e-05 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386433+0000", 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2810000000000008e-06 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477976+0000", 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091542739999999997 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478069+0000", 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2971000000000001e-05 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478237+0000", 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167949 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106205 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.425947791999999, 2024-09-22T01:02:56.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092018127000000005, 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106205 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387017+0000", 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056957000000000004 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387024+0000", 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387035+0000", 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1187e-05 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387045+0000", 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0750000000000004e-06 2024-09-22T01:02:56.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478259+0000", 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091214394000000004 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478311+0000", 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1984000000000002e-05 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478466+0000", 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015504200000000001 2024-09-22T01:02:56.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106206 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.425332405999999, 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091679640000000007, 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106206 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387533+0000", 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047020100000000001 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387539+0000", 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8200000000000002e-06 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387552+0000", 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2865e-05 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387560+0000", 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5499999999999997e-06 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478507+0000", 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090947741999999998 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478599+0000", 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1181000000000007e-05 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478743+0000", 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014428100000000001 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106208 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.424783603000002, 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091664238999999995, 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:56.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106208 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:56.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391033+0000", 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00342144 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391040+0000", 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391054+0000", 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4089e-05 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391063+0000", 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3809999999999999e-06 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479014+0000", 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087951647999999993 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479099+0000", 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4859999999999997e-05 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479276+0000", 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017700699999999999 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:56.909 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:56.910 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:56.911 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:02:56.911 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:56.909+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:57.012 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:57.009+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:57.055 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:02:57.056 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:57.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:02:57.118 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:57.117+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:57.194 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:02:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:02:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:02:57.195 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:57.209 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:02:57.215 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:57.213+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:57.315 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:57.313+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:57.385 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:02:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:02:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908850676, 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:02:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:02:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908829847000007, 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:02:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:02:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908801331999996, 2024-09-22T01:02:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:02:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908778682000005, 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908756686999993, 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:02:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908731622999994, 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908709189999996, 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:02:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908687713000006, 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:02:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.908658955000007, 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.907845963, 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:02:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.907821278, 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903499744000001, 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903470893000005, 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:02:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:02:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903449167999995, 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.447 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903428738000002, 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903406422000003, 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.452 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:02:57.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:02:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:02:57.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903338626999997, 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903305489999994, 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:57.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903281567999997, 2024-09-22T01:02:57.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.468 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.903257971000002, 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:02:57.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:02:57.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:02:57.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:57.473 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:57.473 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:57.474 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:02:57.474 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:57.473+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:57.575 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:57.573+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:57.605 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:57.606 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:57.606 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:57.606 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:57.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:02:57.676 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:57.673+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:57.777 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:57.777+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:02:57.793 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:57.793 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:57.793 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:57.794 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:57.794 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:57.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:02:57.878 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:57.877+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:57.909 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:02:57.978 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:57.978 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006941807, 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.982 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006889284, 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:02:57.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:02:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006836963, 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:57.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:57.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:57.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:57.990 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006671189, 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:57.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.994 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.00664276099999, 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:57.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:57.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.00661495200001, 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:57.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:02:58.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:02:58.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.00658684699999, 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.002 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:02:58.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:02:58.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006559305, 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:02:58.006 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006531657, 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:02:58.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:02:58.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:02:58.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.00650451200001, 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.014 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006476553, 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.018 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:02:58.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006448738, 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:02:58.022 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006421793, 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.026 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:02:58.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.00639409199999, 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.030 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:02:58.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.034 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006279668, 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:02:58.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.038 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.00624031, 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:02:58.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.00621015500001, 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.042 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:58.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:02:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:02:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.00618089699999, 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.046 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006153021, 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:02:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:02:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.006123647, 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:02:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:02:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:02:58.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.057 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.058 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:58.058 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:57.977+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:58.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:02:58.079 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:02:58.076+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:02:58.082 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:02:58.083 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:02:58.083 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-22T01:02:58.083 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:02:58.181 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:58.181+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:58.186 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:58.186 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:58.200 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:02:58.282 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:58.276+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:58.369 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:02:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:02:58.369 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:58.383 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:02:58.383 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:58.381+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:58.388 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 132, 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-22T01:00:18.015746+0000", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-22T01:02:57.723986+0000", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-22T01:01:26.960327+0000", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-22T01:02:34.026131+0000", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5799936, 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-22T01:02:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-22T01:02:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:30.699559+0000", 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-22T01:02:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:36.410605+0000", 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 2, 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 21, 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9900000095367432, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9900000095367432, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:02:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:46.342370+0000", 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:02:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:02:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "132", 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 50, 2024-09-22T01:02:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 132, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:02:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:02:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.7400000095367432, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.7400000095367432, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-22T01:02:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:02:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "8c0f94a8-3c63-442d-9dc1-50c1278929da", 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 124, 2024-09-22T01:02:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-22T01:02:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2277223948", 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2277223948", 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2277223948", 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2277223948", 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "2d2cf30e-e9bd-40ba-8293-193faa02ea69", 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 113, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/463340194", 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/463340194", 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/463340194", 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/463340194", 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a3d3e330-9ea1-4a5c-811b-f285f9a3cff4", 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 112, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/387643397", 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/387643397", 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/387643397", 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/387643397", 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "239072f0-4b69-4624-ba6c-733f299208da", 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 113, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1215210764", 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1215210764", 2024-09-22T01:02:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1215210764", 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1215210764", 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fd35cb24-eb2c-4c17-abe9-ce5c34db9014", 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 0, 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0, 2024-09-22T01:02:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 54, 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6800", 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6801", 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6802", 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6803", 2024-09-22T01:02:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6806", 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6807", 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6804", 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6805", 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6801/2463696152", 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6803/2463696152", 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6807/2463696152", 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6805/2463696152", 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "b60a584d-5334-41e4-92d9-f6689de6b8bf", 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 15, 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 25, 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 52, 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 113, 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 26, 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6824", 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6825", 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6826", 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6827", 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6830", 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6831", 2024-09-22T01:02:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6828", 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6829", 2024-09-22T01:02:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6825/1737475131", 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6827/1737475131", 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1737475131", 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1737475131", 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "9be24894-1100-4e4e-bf1b-79e163fd97e0", 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 125, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6816", 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6817", 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6818", 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6819", 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6822", 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6823", 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6820", 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6821", 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6817/4220217922", 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6819/4220217922", 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6823/4220217922", 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6821/4220217922", 2024-09-22T01:02:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "293cfbc3-8c2e-40b5-ab89-0a124cc8651c", 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:02:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 112, 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6808", 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6809", 2024-09-22T01:02:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6810", 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6811", 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6814", 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6815", 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6812", 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6813", 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6809/3867156106", 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6811/3867156106", 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3867156106", 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3867156106", 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.556128+0000", 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:02:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.159753+0000", 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.766199+0000", 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.155529+0000", 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 65536, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.226232+0000", 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.243346+0000", 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 26 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.143084+0000", 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:02:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.524369+0000", 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.17", 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: 1 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [ 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.13", 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 7, 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 3 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-22T01:02:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [ 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 38, 2024-09-22T01:02:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 11 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [ 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 38, 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 1 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [ 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 49, 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 2 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-22T01:02:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-22T01:02:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-22T01:02:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-22T01:02:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:02:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:02:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-22T01:02:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:02:58.464 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:02:58.466 INFO:tasks.thrashosds.thrasher:j is {'epoch': 132, 'fsid': '7d08b428-4185-4f0f-a0fc-f7dc1fefdce9', 'created': '2024-09-22T01:00:18.015746+0000', 'modified': '2024-09-22T01:02:57.723986+0000', 'last_up_change': '2024-09-22T01:01:26.960327+0000', 'last_in_change': '2024-09-22T01:02:34.026131+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:00:30.699559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:00:36.410605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:00:46.342370+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 132, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8c0f94a8-3c63-442d-9dc1-50c1278929da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 124, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2277223948}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2277223948}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2277223948}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2277223948}]}, 'public_addr': '172.21.15.2:6825/2277223948', 'cluster_addr': '172.21.15.2:6827/2277223948', 'heartbeat_back_addr': '172.21.15.2:6831/2277223948', 'heartbeat_front_addr': '172.21.15.2:6829/2277223948', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 463340194}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 463340194}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 463340194}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 463340194}]}, 'public_addr': '172.21.15.2:6817/463340194', 'cluster_addr': '172.21.15.2:6819/463340194', 'heartbeat_back_addr': '172.21.15.2:6823/463340194', 'heartbeat_front_addr': '172.21.15.2:6821/463340194', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 387643397}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 387643397}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 387643397}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 387643397}]}, 'public_addr': '172.21.15.2:6801/387643397', 'cluster_addr': '172.21.15.2:6803/387643397', 'heartbeat_back_addr': '172.21.15.2:6807/387643397', 'heartbeat_front_addr': '172.21.15.2:6805/387643397', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '239072f0-4b69-4624-ba6c-733f299208da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1215210764}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1215210764}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1215210764}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1215210764}]}, 'public_addr': '172.21.15.2:6809/1215210764', 'cluster_addr': '172.21.15.2:6811/1215210764', 'heartbeat_back_addr': '172.21.15.2:6815/1215210764', 'heartbeat_front_addr': '172.21.15.2:6813/1215210764', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 2463696152}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 2463696152}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 2463696152}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 2463696152}]}, 'public_addr': '172.21.15.119:6801/2463696152', 'cluster_addr': '172.21.15.119:6803/2463696152', 'heartbeat_back_addr': '172.21.15.119:6807/2463696152', 'heartbeat_front_addr': '172.21.15.119:6805/2463696152', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b60a584d-5334-41e4-92d9-f6689de6b8bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 25, 'up_from': 52, 'up_thru': 113, 'down_at': 26, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1737475131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1737475131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1737475131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1737475131}]}, 'public_addr': '172.21.15.119:6825/1737475131', 'cluster_addr': '172.21.15.119:6827/1737475131', 'heartbeat_back_addr': '172.21.15.119:6831/1737475131', 'heartbeat_front_addr': '172.21.15.119:6829/1737475131', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9be24894-1100-4e4e-bf1b-79e163fd97e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 4220217922}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 4220217922}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 4220217922}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 4220217922}]}, 'public_addr': '172.21.15.119:6817/4220217922', 'cluster_addr': '172.21.15.119:6819/4220217922', 'heartbeat_back_addr': '172.21.15.119:6823/4220217922', 'heartbeat_front_addr': '172.21.15.119:6821/4220217922', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '293cfbc3-8c2e-40b5-ab89-0a124cc8651c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3867156106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3867156106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3867156106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3867156106}]}, 'public_addr': '172.21.15.119:6809/3867156106', 'cluster_addr': '172.21.15.119:6811/3867156106', 'heartbeat_back_addr': '172.21.15.119:6815/3867156106', 'heartbeat_front_addr': '172.21.15.119:6813/3867156106', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.556128+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.159753+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.766199+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.155529+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.226232+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.243346+0000', 'dead_epoch': 26}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.143084+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.524369+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.17', 'osds': [6, 1]}], 'pg_upmap_items': [{'pgid': '3.13', 'mappings': [{'from': 7, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 38, 'length': 11}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 38, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 49, 'length': 2}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:02:58.466 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:02:58.506 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:02:58.504+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:02:58.585 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:58.585+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235919542999994, 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:02:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235882574000001, 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235854305000004, 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:02:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:02:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235816428999996, 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:02:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:02:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235777838999994, 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235705511000006, 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:02:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:02:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235667645000007, 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235625405999997, 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:02:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:02:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235571020999998, 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:02:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235470255999999, 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:02:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235383330999994, 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:02:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:02:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:02:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.235325007, 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:02:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:02:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.234871857000002, 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:02:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:02:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.234233371000002, 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:02:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.233945774000006, 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:02:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.233342805000007, 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:02:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:02:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.232262505999998, 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:02:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:02:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:02:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.232192628000007, 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:02:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.231630636999995, 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:02:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:02:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.231253211999999, 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:02:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:02:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:02:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:02:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:02:58.683 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:58.684 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:02:58.685 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:58.680+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:58.714 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:02:58.787 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:58.785+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:58.821 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:02:58.822 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:02:58.841 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-22T01:02:58.656051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1080721277,"num_objects":1433,"num_object_clones":402,"num_object_copies":2866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":594,"num_read":193594,"num_read_kb":752201,"num_write":21095,"num_write_kb":413075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2893,"num_bytes_recovered":182030025,"num_keys_recovered":784,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17411,"ondisk_log_size":17411,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":649972,"kb_used_data":302836,"kb_used_omap":13,"kb_used_meta":346930,"kb_avail":659952908,"statfs":{"total":676457349120,"available":675791777792,"internally_reserved":0,"allocated":310104064,"data_stored":373435756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14092,"internal_metadata":355256564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-263127840,"num_objects":-135,"num_object_clones":-36,"num_object_copies":-270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-135,"num_whiteouts":-13,"num_read":1308,"num_read_kb":1124,"num_write":267,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000965"},"pg_stats":[{"pgid":"3.1f","version":"129'477","reported_seq":6817,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.031786+0000","last_change":"2024-09-22T01:02:54.944660+0000","last_active":"2024-09-22T01:02:55.031786+0000","last_peered":"2024-09-22T01:02:55.031786+0000","last_clean":"2024-09-22T01:02:55.031786+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:02:55.031786+0000","last_undegraded":"2024-09-22T01:02:55.031786+0000","last_fullsized":"2024-09-22T01:02:55.031786+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:06:53.895175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017843589999999999,"stat_sum":{"num_bytes":34927270,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":6502,"num_read_kb":45938,"num_write":523,"num_write_kb":17793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.1e","version":"128'664","reported_seq":3116,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.894292+0000","last_change":"2024-09-22T01:02:54.895283+0000","last_active":"2024-09-22T01:02:54.894292+0000","last_peered":"2024-09-22T01:02:54.894292+0000","last_clean":"2024-09-22T01:02:54.894292+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:02:54.894292+0000","last_undegraded":"2024-09-22T01:02:54.894292+0000","last_fullsized":"2024-09-22T01:02:54.894292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:20:36.125908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025063569999999999,"stat_sum":{"num_bytes":28358594,"num_objects":57,"num_object_clones":16,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":22,"num_read":5885,"num_read_kb":12587,"num_write":903,"num_write_kb":7183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.1d","version":"129'428","reported_seq":4598,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.905230+0000","last_change":"2024-09-22T01:02:55.905842+0000","last_active":"2024-09-22T01:02:55.905230+0000","last_peered":"2024-09-22T01:02:55.905230+0000","last_clean":"2024-09-22T01:02:55.905230+0000","last_became_active":"2024-09-22T01:01:43.138236+0000","last_became_peered":"2024-09-22T01:01:43.138236+0000","last_unstale":"2024-09-22T01:02:55.905230+0000","last_undegraded":"2024-09-22T01:02:55.905230+0000","last_fullsized":"2024-09-22T01:02:55.905230+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:48:52.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002094378,"stat_sum":{"num_bytes":57068431,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":4787,"num_read_kb":32328,"num_write":467,"num_write_kb":18066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":7734152,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.1c","version":"130'1006","reported_seq":9857,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.943948+0000","last_change":"2024-09-22T01:02:55.904378+0000","last_active":"2024-09-22T01:02:55.943948+0000","last_peered":"2024-09-22T01:02:55.943948+0000","last_clean":"2024-09-22T01:02:55.943948+0000","last_became_active":"2024-09-22T01:02:39.738124+0000","last_became_peered":"2024-09-22T01:02:39.738124+0000","last_unstale":"2024-09-22T01:02:55.943948+0000","last_undegraded":"2024-09-22T01:02:55.943948+0000","last_fullsized":"2024-09-22T01:02:55.943948+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1006,"log_dups_size":0,"ondisk_log_size":1006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:03:58.229280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013246779999999999,"stat_sum":{"num_bytes":61399102,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":19,"num_read":11219,"num_read_kb":51469,"num_write":1492,"num_write_kb":18274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":31958297,"num_keys_recovered":274,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.1b","version":"127'1037","reported_seq":2513,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.942814+0000","last_change":"2024-09-22T01:02:54.943559+0000","last_active":"2024-09-22T01:02:54.942814+0000","last_peered":"2024-09-22T01:02:54.942814+0000","last_clean":"2024-09-22T01:02:54.942814+0000","last_became_active":"2024-09-22T01:01:43.145371+0000","last_became_peered":"2024-09-22T01:01:43.145371+0000","last_unstale":"2024-09-22T01:02:54.942814+0000","last_undegraded":"2024-09-22T01:02:54.942814+0000","last_fullsized":"2024-09-22T01:02:54.942814+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:32.825203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010262979999999999,"stat_sum":{"num_bytes":26640182,"num_objects":44,"num_object_clones":16,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":5158,"num_read_kb":15383,"num_write":1492,"num_write_kb":8379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.1a","version":"127'445","reported_seq":11763,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.064172+0000","last_change":"2024-09-22T01:02:54.884931+0000","last_active":"2024-09-22T01:02:55.064172+0000","last_peered":"2024-09-22T01:02:55.064172+0000","last_clean":"2024-09-22T01:02:55.064172+0000","last_became_active":"2024-09-22T01:02:39.738141+0000","last_became_peered":"2024-09-22T01:02:39.738141+0000","last_unstale":"2024-09-22T01:02:55.064172+0000","last_undegraded":"2024-09-22T01:02:55.064172+0000","last_fullsized":"2024-09-22T01:02:55.064172+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:08:51.781515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673262,"stat_sum":{"num_bytes":37756919,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":11889,"num_read_kb":25736,"num_write":524,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":7836182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.19","version":"129'358","reported_seq":9711,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.083320+0000","last_change":"2024-09-22T01:02:54.884064+0000","last_active":"2024-09-22T01:02:55.083320+0000","last_peered":"2024-09-22T01:02:55.083320+0000","last_clean":"2024-09-22T01:02:55.083320+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:02:55.083320+0000","last_undegraded":"2024-09-22T01:02:55.083320+0000","last_fullsized":"2024-09-22T01:02:55.083320+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:57.009779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019429440000000001,"stat_sum":{"num_bytes":28461919,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":9097,"num_read_kb":31278,"num_write":429,"num_write_kb":9304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.18","version":"127'654","reported_seq":5934,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.893339+0000","last_change":"2024-09-22T01:02:54.894218+0000","last_active":"2024-09-22T01:02:54.893339+0000","last_peered":"2024-09-22T01:02:54.893339+0000","last_clean":"2024-09-22T01:02:54.893339+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:02:54.893339+0000","last_undegraded":"2024-09-22T01:02:54.893339+0000","last_fullsized":"2024-09-22T01:02:54.893339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:06.140587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017521469999999999,"stat_sum":{"num_bytes":76308676,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":14,"num_read":7979,"num_read_kb":42181,"num_write":760,"num_write_kb":16828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.17","version":"130'471","reported_seq":3844,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.894285+0000","last_change":"2024-09-22T01:02:55.894285+0000","last_active":"2024-09-22T01:02:55.894285+0000","last_peered":"2024-09-22T01:02:55.894285+0000","last_clean":"2024-09-22T01:02:55.894285+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:02:55.894285+0000","last_undegraded":"2024-09-22T01:02:55.894285+0000","last_fullsized":"2024-09-22T01:02:55.894285+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:39:45.193736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029832190000000001,"stat_sum":{"num_bytes":15872880,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":20,"num_read":4518,"num_read_kb":18058,"num_write":577,"num_write_kb":8510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.16","version":"129'330","reported_seq":1913,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.904864+0000","last_change":"2024-09-22T01:02:55.905361+0000","last_active":"2024-09-22T01:02:55.904864+0000","last_peered":"2024-09-22T01:02:55.904864+0000","last_clean":"2024-09-22T01:02:55.904864+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:02:55.904864+0000","last_undegraded":"2024-09-22T01:02:55.904864+0000","last_fullsized":"2024-09-22T01:02:55.904864+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:46:36.490144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001987117,"stat_sum":{"num_bytes":65625308,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":2571,"num_read_kb":34280,"num_write":374,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.15","version":"129'579","reported_seq":6265,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.047824+0000","last_change":"2024-09-22T01:02:54.889838+0000","last_active":"2024-09-22T01:02:55.047824+0000","last_peered":"2024-09-22T01:02:55.047824+0000","last_clean":"2024-09-22T01:02:55.047824+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:02:55.047824+0000","last_undegraded":"2024-09-22T01:02:55.047824+0000","last_fullsized":"2024-09-22T01:02:55.047824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:27:13.402354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001828507,"stat_sum":{"num_bytes":27796518,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":8369,"num_read_kb":26687,"num_write":617,"num_write_kb":5744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.14","version":"126'474","reported_seq":13479,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.888118+0000","last_change":"2024-09-22T01:02:54.888479+0000","last_active":"2024-09-22T01:02:54.888118+0000","last_peered":"2024-09-22T01:02:54.888118+0000","last_clean":"2024-09-22T01:02:54.888118+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:02:54.888118+0000","last_undegraded":"2024-09-22T01:02:54.888118+0000","last_fullsized":"2024-09-22T01:02:54.888118+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:49:44.923758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064133300000000001,"stat_sum":{"num_bytes":47464584,"num_objects":48,"num_object_clones":16,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":13638,"num_read_kb":40761,"num_write":512,"num_write_kb":17919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.890854+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:02:55.890854+0000","last_peered":"2024-09-22T01:02:55.890854+0000","last_clean":"2024-09-22T01:02:55.890854+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:02:55.890854+0000","last_undegraded":"2024-09-22T01:02:55.890854+0000","last_fullsized":"2024-09-22T01:02:55.890854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"129'471","reported_seq":2597,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.943688+0000","last_change":"2024-09-22T01:02:54.943688+0000","last_active":"2024-09-22T01:02:54.943688+0000","last_peered":"2024-09-22T01:02:54.943688+0000","last_clean":"2024-09-22T01:02:54.943688+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:02:54.943688+0000","last_undegraded":"2024-09-22T01:02:54.943688+0000","last_fullsized":"2024-09-22T01:02:54.943688+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:58.523306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001110099,"stat_sum":{"num_bytes":16578718,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":3645,"num_read_kb":6134,"num_write":486,"num_write_kb":3309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.883391+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:02:54.883391+0000","last_peered":"2024-09-22T01:02:54.883391+0000","last_clean":"2024-09-22T01:02:54.883391+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:02:54.883391+0000","last_undegraded":"2024-09-22T01:02:54.883391+0000","last_fullsized":"2024-09-22T01:02:54.883391+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"127'531","reported_seq":3284,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.895509+0000","last_change":"2024-09-22T01:02:54.896055+0000","last_active":"2024-09-22T01:02:54.895509+0000","last_peered":"2024-09-22T01:02:54.895509+0000","last_clean":"2024-09-22T01:02:54.895509+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:02:54.895509+0000","last_undegraded":"2024-09-22T01:02:54.895509+0000","last_fullsized":"2024-09-22T01:02:54.895509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:43:48.710411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029446910000000001,"stat_sum":{"num_bytes":33651141,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":4460,"num_read_kb":12694,"num_write":670,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":236,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.888217+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:02:54.888217+0000","last_peered":"2024-09-22T01:02:54.888217+0000","last_clean":"2024-09-22T01:02:54.888217+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:02:54.888217+0000","last_undegraded":"2024-09-22T01:02:54.888217+0000","last_fullsized":"2024-09-22T01:02:54.888217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"128'387","reported_seq":4052,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.885089+0000","last_change":"2024-09-22T01:02:54.885870+0000","last_active":"2024-09-22T01:02:54.885089+0000","last_peered":"2024-09-22T01:02:54.885089+0000","last_clean":"2024-09-22T01:02:54.885089+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:02:54.885089+0000","last_undegraded":"2024-09-22T01:02:54.885089+0000","last_fullsized":"2024-09-22T01:02:54.885089+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:00:15.560911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021894990000000001,"stat_sum":{"num_bytes":15107076,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":3800,"num_read_kb":26304,"num_write":386,"num_write_kb":16759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.883186+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:02:54.883186+0000","last_peered":"2024-09-22T01:02:54.883186+0000","last_clean":"2024-09-22T01:02:54.883186+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:02:54.883186+0000","last_undegraded":"2024-09-22T01:02:54.883186+0000","last_fullsized":"2024-09-22T01:02:54.883186+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"129'554","reported_seq":3886,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.904524+0000","last_change":"2024-09-22T01:02:55.905080+0000","last_active":"2024-09-22T01:02:55.904524+0000","last_peered":"2024-09-22T01:02:55.904524+0000","last_clean":"2024-09-22T01:02:55.904524+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:02:55.904524+0000","last_undegraded":"2024-09-22T01:02:55.904524+0000","last_fullsized":"2024-09-22T01:02:55.904524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:06:40.856622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017398590000000001,"stat_sum":{"num_bytes":46029364,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":6326,"num_read_kb":12163,"num_write":505,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.3","version":"129'513","reported_seq":2249,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.058111+0000","last_change":"2024-09-22T01:02:54.944576+0000","last_active":"2024-09-22T01:02:55.058111+0000","last_peered":"2024-09-22T01:02:55.058111+0000","last_clean":"2024-09-22T01:02:55.058111+0000","last_became_active":"2024-09-22T01:01:43.147440+0000","last_became_peered":"2024-09-22T01:01:43.147440+0000","last_unstale":"2024-09-22T01:02:55.058111+0000","last_undegraded":"2024-09-22T01:02:55.058111+0000","last_fullsized":"2024-09-22T01:02:55.058111+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:26.389242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001705563,"stat_sum":{"num_bytes":19383445,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":2954,"num_read_kb":17598,"num_write":558,"num_write_kb":15103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"2.2","version":"21'2","reported_seq":210,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.902722+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:02:55.902722+0000","last_peered":"2024-09-22T01:02:55.902722+0000","last_clean":"2024-09-22T01:02:55.902722+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:02:55.902722+0000","last_undegraded":"2024-09-22T01:02:55.902722+0000","last_fullsized":"2024-09-22T01:02:55.902722+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"127'1568","reported_seq":3454,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.883717+0000","last_change":"2024-09-22T01:02:54.885215+0000","last_active":"2024-09-22T01:02:54.883717+0000","last_peered":"2024-09-22T01:02:54.883717+0000","last_clean":"2024-09-22T01:02:54.883717+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:02:54.883717+0000","last_undegraded":"2024-09-22T01:02:54.883717+0000","last_fullsized":"2024-09-22T01:02:54.883717+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1568,"log_dups_size":0,"ondisk_log_size":1568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:45:34.873506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027822279999999999,"stat_sum":{"num_bytes":32604432,"num_objects":47,"num_object_clones":15,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":8870,"num_read_kb":25285,"num_write":2647,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":236,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.880764+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:02:54.880764+0000","last_peered":"2024-09-22T01:02:54.880764+0000","last_clean":"2024-09-22T01:02:54.880764+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:02:54.880764+0000","last_undegraded":"2024-09-22T01:02:54.880764+0000","last_fullsized":"2024-09-22T01:02:54.880764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"128'346","reported_seq":2581,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.944024+0000","last_change":"2024-09-22T01:02:54.944024+0000","last_active":"2024-09-22T01:02:54.944024+0000","last_peered":"2024-09-22T01:02:54.944024+0000","last_clean":"2024-09-22T01:02:54.944024+0000","last_became_active":"2024-09-22T01:01:43.151452+0000","last_became_peered":"2024-09-22T01:01:43.151452+0000","last_unstale":"2024-09-22T01:02:54.944024+0000","last_undegraded":"2024-09-22T01:02:54.944024+0000","last_fullsized":"2024-09-22T01:02:54.944024+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:14.483879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001349538,"stat_sum":{"num_bytes":40916259,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":2187,"num_read_kb":18579,"num_write":372,"num_write_kb":15217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1440739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":246,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.887634+0000","last_change":"2024-09-22T01:02:35.049954+0000","last_active":"2024-09-22T01:02:54.887634+0000","last_peered":"2024-09-22T01:02:54.887634+0000","last_clean":"2024-09-22T01:02:54.887634+0000","last_became_active":"2024-09-22T01:02:35.049024+0000","last_became_peered":"2024-09-22T01:02:35.049024+0000","last_unstale":"2024-09-22T01:02:54.887634+0000","last_undegraded":"2024-09-22T01:02:54.887634+0000","last_fullsized":"2024-09-22T01:02:54.887634+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:45.992332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"127'341","reported_seq":2014,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.895168+0000","last_change":"2024-09-22T01:02:54.895168+0000","last_active":"2024-09-22T01:02:54.895168+0000","last_peered":"2024-09-22T01:02:54.895168+0000","last_clean":"2024-09-22T01:02:54.895168+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:02:54.895168+0000","last_undegraded":"2024-09-22T01:02:54.895168+0000","last_fullsized":"2024-09-22T01:02:54.895168+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:33.060216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022983769999999999,"stat_sum":{"num_bytes":24411263,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":1621,"num_read_kb":15577,"num_write":349,"num_write_kb":17862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"2.3","version":"19'1","reported_seq":191,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.902764+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:02:55.902764+0000","last_peered":"2024-09-22T01:02:55.902764+0000","last_clean":"2024-09-22T01:02:55.902764+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:02:55.902764+0000","last_undegraded":"2024-09-22T01:02:55.902764+0000","last_fullsized":"2024-09-22T01:02:55.902764+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":387,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.887641+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:02:54.887641+0000","last_peered":"2024-09-22T01:02:54.887641+0000","last_clean":"2024-09-22T01:02:54.887641+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:02:54.887641+0000","last_undegraded":"2024-09-22T01:02:54.887641+0000","last_fullsized":"2024-09-22T01:02:54.887641+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"128'378","reported_seq":10117,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.893508+0000","last_change":"2024-09-22T01:02:54.894115+0000","last_active":"2024-09-22T01:02:54.893508+0000","last_peered":"2024-09-22T01:02:54.893508+0000","last_clean":"2024-09-22T01:02:54.893508+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:02:54.893508+0000","last_undegraded":"2024-09-22T01:02:54.893508+0000","last_fullsized":"2024-09-22T01:02:54.893508+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:16.199050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016074259999999999,"stat_sum":{"num_bytes":46892985,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":10373,"num_read_kb":27112,"num_write":386,"num_write_kb":13691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.9","version":"127'507","reported_seq":2008,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.881486+0000","last_change":"2024-09-22T01:02:54.882205+0000","last_active":"2024-09-22T01:02:54.881486+0000","last_peered":"2024-09-22T01:02:54.881486+0000","last_clean":"2024-09-22T01:02:54.881486+0000","last_became_active":"2024-09-22T01:01:43.141045+0000","last_became_peered":"2024-09-22T01:01:43.141045+0000","last_unstale":"2024-09-22T01:02:54.881486+0000","last_undegraded":"2024-09-22T01:02:54.881486+0000","last_fullsized":"2024-09-22T01:02:54.881486+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:20:17.914473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011839680000000001,"stat_sum":{"num_bytes":28740814,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":2597,"num_read_kb":6972,"num_write":594,"num_write_kb":8014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":156591,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.a","version":"130'698","reported_seq":15465,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:56.036521+0000","last_change":"2024-09-22T01:02:55.894077+0000","last_active":"2024-09-22T01:02:56.036521+0000","last_peered":"2024-09-22T01:02:56.036521+0000","last_clean":"2024-09-22T01:02:56.036521+0000","last_became_active":"2024-09-22T01:02:37.737218+0000","last_became_peered":"2024-09-22T01:02:37.737218+0000","last_unstale":"2024-09-22T01:02:56.036521+0000","last_undegraded":"2024-09-22T01:02:56.036521+0000","last_fullsized":"2024-09-22T01:02:56.036521+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:03:00.328184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131123,"stat_sum":{"num_bytes":22965762,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":16346,"num_read_kb":40328,"num_write":743,"num_write_kb":13027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":13401696,"num_keys_recovered":167,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.b","version":"123'598","reported_seq":2366,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.896185+0000","last_change":"2024-09-22T01:02:54.896877+0000","last_active":"2024-09-22T01:02:54.896185+0000","last_peered":"2024-09-22T01:02:54.896185+0000","last_clean":"2024-09-22T01:02:54.896185+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:02:54.896185+0000","last_undegraded":"2024-09-22T01:02:54.896185+0000","last_fullsized":"2024-09-22T01:02:54.896185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:26.409026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032024839999999998,"stat_sum":{"num_bytes":19277367,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":4005,"num_read_kb":19600,"num_write":749,"num_write_kb":10850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.c","version":"126'363","reported_seq":1115,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.904050+0000","last_change":"2024-09-22T01:02:55.904682+0000","last_active":"2024-09-22T01:02:55.904050+0000","last_peered":"2024-09-22T01:02:55.904050+0000","last_clean":"2024-09-22T01:02:55.904050+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:02:55.904050+0000","last_undegraded":"2024-09-22T01:02:55.904050+0000","last_fullsized":"2024-09-22T01:02:55.904050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:34:42.204457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014764509999999999,"stat_sum":{"num_bytes":27120964,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":2557,"num_read_kb":21657,"num_write":372,"num_write_kb":11071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.d","version":"129'666","reported_seq":5453,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.041438+0000","last_change":"2024-09-22T01:02:54.889972+0000","last_active":"2024-09-22T01:02:55.041438+0000","last_peered":"2024-09-22T01:02:55.041438+0000","last_clean":"2024-09-22T01:02:55.041438+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:02:55.041438+0000","last_undegraded":"2024-09-22T01:02:55.041438+0000","last_fullsized":"2024-09-22T01:02:55.041438+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:26:00.129782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018152159999999999,"stat_sum":{"num_bytes":31107082,"num_objects":54,"num_object_clones":18,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":7056,"num_read_kb":28743,"num_write":844,"num_write_kb":17616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.e","version":"129'426","reported_seq":4568,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.888868+0000","last_change":"2024-09-22T01:02:54.888868+0000","last_active":"2024-09-22T01:02:54.888868+0000","last_peered":"2024-09-22T01:02:54.888868+0000","last_clean":"2024-09-22T01:02:54.888868+0000","last_became_active":"2024-09-22T01:02:38.737922+0000","last_became_peered":"2024-09-22T01:02:38.737922+0000","last_unstale":"2024-09-22T01:02:54.888868+0000","last_undegraded":"2024-09-22T01:02:54.888868+0000","last_fullsized":"2024-09-22T01:02:54.888868+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:34:35.400807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094628599999999996,"stat_sum":{"num_bytes":39243498,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":3792,"num_read_kb":26242,"num_write":486,"num_write_kb":15956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":14730069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.f","version":"128'343","reported_seq":4514,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.888640+0000","last_change":"2024-09-22T01:02:54.888991+0000","last_active":"2024-09-22T01:02:54.888640+0000","last_peered":"2024-09-22T01:02:54.888640+0000","last_clean":"2024-09-22T01:02:54.888640+0000","last_became_active":"2024-09-22T01:01:43.142069+0000","last_became_peered":"2024-09-22T01:01:43.142069+0000","last_unstale":"2024-09-22T01:02:54.888640+0000","last_undegraded":"2024-09-22T01:02:54.888640+0000","last_fullsized":"2024-09-22T01:02:54.888640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:23:15.958606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010538940000000001,"stat_sum":{"num_bytes":22207174,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":4998,"num_read_kb":10197,"num_write":287,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":621318,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.10","version":"127'386","reported_seq":2927,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-22T01:02:55.891336+0000","last_change":"2024-09-22T01:02:55.891898+0000","last_active":"2024-09-22T01:02:55.891336+0000","last_peered":"2024-09-22T01:02:55.891336+0000","last_clean":"2024-09-22T01:02:55.891336+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:02:55.891336+0000","last_undegraded":"2024-09-22T01:02:55.891336+0000","last_fullsized":"2024-09-22T01:02:55.891336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:18.920951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005045,"stat_sum":{"num_bytes":38981771,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":27,"num_read":2999,"num_read_kb":14514,"num_write":353,"num_write_kb":12727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.11","version":"128'609","reported_seq":2583,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.888202+0000","last_change":"2024-09-22T01:02:54.888550+0000","last_active":"2024-09-22T01:02:54.888202+0000","last_peered":"2024-09-22T01:02:54.888202+0000","last_clean":"2024-09-22T01:02:54.888202+0000","last_became_active":"2024-09-22T01:01:43.138994+0000","last_became_peered":"2024-09-22T01:01:43.138994+0000","last_unstale":"2024-09-22T01:02:54.888202+0000","last_undegraded":"2024-09-22T01:02:54.888202+0000","last_fullsized":"2024-09-22T01:02:54.888202+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:51.357252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063930499999999995,"stat_sum":{"num_bytes":24564457,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":4306,"num_read_kb":11552,"num_write":761,"num_write_kb":5783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1134106,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.12","version":"125'386","reported_seq":2791,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.942736+0000","last_change":"2024-09-22T01:02:54.943237+0000","last_active":"2024-09-22T01:02:54.942736+0000","last_peered":"2024-09-22T01:02:54.942736+0000","last_clean":"2024-09-22T01:02:54.942736+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:02:54.942736+0000","last_undegraded":"2024-09-22T01:02:54.942736+0000","last_fullsized":"2024-09-22T01:02:54.942736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:23.472571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078729400000000001,"stat_sum":{"num_bytes":24020824,"num_objects":45,"num_object_clones":15,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":3283,"num_read_kb":7295,"num_write":381,"num_write_kb":8141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"a"}]},{"pgid":"3.13","version":"127'297","reported_seq":6354,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-22T01:02:54.895325+0000","last_change":"2024-09-22T01:02:54.896167+0000","last_active":"2024-09-22T01:02:54.895325+0000","last_peered":"2024-09-22T01:02:54.895325+0000","last_clean":"2024-09-22T01:02:54.895325+0000","last_became_active":"2024-09-22T01:02:06.799165+0000","last_became_peered":"2024-09-22T01:02:06.799165+0000","last_unstale":"2024-09-22T01:02:54.895325+0000","last_undegraded":"2024-09-22T01:02:54.895325+0000","last_fullsized":"2024-09-22T01:02:54.895325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:42.737847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031345779999999998,"stat_sum":{"num_bytes":18646111,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":5691,"num_read_kb":26869,"num_write":335,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":11755308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1080130890,"num_objects":1429,"num_object_clones":402,"num_object_copies":2858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1429,"num_whiteouts":594,"num_read":193478,"num_read_kb":752101,"num_write":20934,"num_write_kb":410657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2891,"num_bytes_recovered":182029987,"num_keys_recovered":784,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":324931584,"data_stored":398098009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":17291,"ondisk_log_size":17291,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64824,"kb_used_data":37788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94307016,"statfs":{"total":96636764160,"available":96570384384,"internally_reserved":0,"allocated":38694912,"data_stored":47031498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299411,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98884,"kb_used_data":58280,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94272956,"statfs":{"total":96636764160,"available":96535506944,"internally_reserved":0,"allocated":59678720,"data_stored":78865351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":41545294},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91780,"kb_used_data":38368,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94280060,"statfs":{"total":96636764160,"available":96542781440,"internally_reserved":0,"allocated":39288832,"data_stored":46781025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542177,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118768,"kb_used_data":61212,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94253072,"statfs":{"total":96636764160,"available":96515145728,"internally_reserved":0,"allocated":62681088,"data_stored":74968319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542176,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69428,"kb_used_data":20192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94302412,"statfs":{"total":96636764160,"available":96565669888,"internally_reserved":0,"allocated":20676608,"data_stored":24256705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106952,"kb_used_data":45228,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94264888,"statfs":{"total":96636764160,"available":96527245312,"internally_reserved":0,"allocated":46313472,"data_stored":54465106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542177,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99336,"kb_used_data":41768,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94272504,"statfs":{"total":96636764160,"available":96535044096,"internally_reserved":0,"allocated":42770432,"data_stored":47067752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40640512,"data_stored":45750716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44773376,"data_stored":53738419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19140608,"data_stored":23530037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61145088,"data_stored":74241651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":26791936,"data_stored":30935997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":58138624,"data_stored":78138664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":37146624,"data_stored":46298577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":37154816,"data_stored":45463948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:02:58.843 INFO:tasks.thrashosds.thrasher:Setting 3.11 to [4, 6] 2024-09-22T01:02:58.843 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.11', '4', '6'] 2024-09-22T01:02:58.844 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.11 4 6 2024-09-22T01:02:58.851 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:02:58.851 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:02:58.851 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:02:58.851 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:02:58.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:02:58.869 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:02:58.869 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:02:58.886 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:02:58.886 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:02:58.904 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:02:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:02:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:02:58.909 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:58.909+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:58.929 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:02:58.930 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:02:58.951 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:02:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:02:58.975 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:02:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:02:58.987 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:02:58.984+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:02:58.998 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:02:58.998 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:02:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:02:59.020 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:02:59.037 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:59.038 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:59.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:02:59.097 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:59.097+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:59.169 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:59.170 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:59.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:02:59.189 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:59.184+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:59.290 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:59.288+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:59.351 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:59.351 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:59.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:02:59.390 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:59.388+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:59.487 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:59.487 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:59.491 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:02:59.489+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:02:59.502 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:02:59.592 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:02:59.588+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:02:59.671 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:59.671 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:59.686 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:02:59.693 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:59.693+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:59.794 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:02:59.793+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:02:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:59.836 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:02:59.895 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:02:59.893+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:02:59.913 INFO:teuthology.orchestra.run.smithi002.stderr:set 3.11 pg_upmap mapping to [4,6] 2024-09-22T01:02:59.973 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:02:59.973 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:02:59.989 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:02:59.996 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:02:59.993+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:00.096 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:00.092+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:00.170 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:00.170 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:00.185 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:00.198 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:00.197+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:00.298 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:00.297+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:00.351 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:00.351 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:00.365 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:00.399 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:00.397+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:00.490 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:00.490 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:00.500 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:00.497+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:00.504 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:00.601 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:00.601+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:00.631 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:00.632 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:00.646 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:00.702 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:00.701+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:00.772 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:00.773 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:00.787 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:00.834 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:00.832+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:00.904 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:00.900+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:00.915 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:00.915 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:00.932 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:03:00.937 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:01.004 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:01.001+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:01.040 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:01.105 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:01.100+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:01.131 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:01.131 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:01.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:01.211 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:01.211 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:01.211 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:01.208+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:01.228 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:01.245 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:01.245 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:01.262 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:01.282 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:01.283 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:01.303 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:01.304 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:01.306 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:01.304+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:01.307 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:01.307 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:01.307 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:01.307 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:01.322 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:01.325 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:01.325 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:01.347 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:01.347 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:01.407 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:01.404+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:01.443 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:01.443 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:01.443 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:01.443 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:01.443 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:01.457 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:01.542 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:01.540+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:01.609 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:01.604+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:01.627 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.152415074000004, 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:03:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:03:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:03:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.152394244999996, 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:03:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:03:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.15236573, 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.152343079999994, 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.642 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:03:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.152321084999997, 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:03:01.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:03:01.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.152296020999998, 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:03:01.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.152273588, 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:03:01.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:03:01.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.152252110999996, 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:03:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:03:01.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.152223352999997, 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:03:01.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:03:01.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.668 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.151410361000003, 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:03:01.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.151385676000004, 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.673 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:03:01.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:03:01.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:03:01.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.147064142000005, 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:03:01.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:03:01.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:03:01.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.147035290999995, 2024-09-22T01:03:01.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:03:01.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:03:01.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.147013565999998, 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.146993136000006, 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:03:01.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:03:01.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.146970820000007, 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:03:01.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.146903025, 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:03:01.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.146869887999998, 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:03:01.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:03:01.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.146845966000001, 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.709 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:03:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.146822369000006, 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:01.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:01.717 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:01.719 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:01.719 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:01.716+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:01.820 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:01.817+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:01.851 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:01.851 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:01.851 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:01.851 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:01.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:01.921 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:01.916+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:01.923 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:03:02.022 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:02.021+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:02.041 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:02.042 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:02.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:02.145 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:02.145+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.25065586300001, 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:03:02.222 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:03:02.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.226 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.25060334, 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.230 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250551019, 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250385245, 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:02.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:02.238 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250356817, 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:03:02.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.25032900799999, 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:03:02.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:03:02.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:03:02.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250300903, 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:03:02.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:03:02.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250273361, 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:02.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:03:02.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:03:02.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250245713, 2024-09-22T01:03:02.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:03:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.25021856799999, 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:02.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:03:02.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:03:02.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.263 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250190609, 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.267 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250162794, 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:02.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250135849, 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.250108148, 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.279 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.24999372400001, 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:03:02.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.249954366, 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.284 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:03:02.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:03:02.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.287 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.24992421100001, 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:03:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.24989495299999, 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:03:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.249867077, 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:03:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:03:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.249837703, 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:02.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:03:02.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:03:02.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.304 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.304 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.304 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:02.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:02.306 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:02.305+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:02.406 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:02.404+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:02.476 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:02.476 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:02.476 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:02.476 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:02.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:02.507 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:02.504+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:02.608 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:02.605+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:02.608 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:02.609 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:02.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:02.709 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:02.704+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:02.795 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436382633999997, 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:03:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:03:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436345665000005, 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:03:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:03:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436317396000007, 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436279519999999, 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:03:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:03:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:03:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436240929999997, 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436168601999995, 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436130735999996, 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436088497, 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:03:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.436034112000002, 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:03:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:03:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.435933347000002, 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:03:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:03:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.435846421999997, 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:03:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:03:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.435788098000003, 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:03:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:03:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.435334948000005, 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:03:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:03:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.434696462000005, 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.434408864999995, 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:03:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:03:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:03:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.433805895999996, 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.432725597000001, 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.432655718999996, 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:02.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.432093727999998, 2024-09-22T01:03:02.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:02.877 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.431716303000002, 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:03:02.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:02.881 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:02.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:02.883 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:02.881+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:02.983 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:02.980+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:03.058 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:03.059 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:03.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:03.085 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:03.080+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:03.185 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:03.180+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:03.194 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:03.194 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:03.194 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:03.194 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:03.194 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:03.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:03.286 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:03.284+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:03.333 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:03.333 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:03.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:03.333 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:03.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.333 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:03:03.333 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 151.43972917599999, 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:03:03.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:03:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055731848999997, 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:03:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:03:03.343 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055698543999995, 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.344 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:03:03.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:03:03.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055602031000006, 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:03:03.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055528780000003, 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:03:03.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055463599000007, 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055393477999999, 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:03.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055360113000006, 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:03.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055329690999997, 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:03.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055299384999998, 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.372 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:03.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055268983000005, 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.376 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:03.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055238184000004, 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.380 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:03.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:03:03.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055207710999994, 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.384 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:03:03.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055177119000007, 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:03:03.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:03:03.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055139316999998, 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:03:03.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055106952000003, 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055076632999999, 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:03:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:03:03.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055046740999998, 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:03:03.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.408 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.408 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:03.409 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.055016981999998, 2024-09-22T01:03:03.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:03:03.409 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.054966516999997, 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:03.413 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:03.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:03.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:03:03.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:03:03.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:03.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:03.417 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:03.417 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:03.419 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:03.419 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:03.416+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:03.520 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:03.517+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:03.542 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:03.542 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:03.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:03.574 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:03.574 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:03.591 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:03.591 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:03.607 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:03.607 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:03.621 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:03.616+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:03.624 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:03.624 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:03.641 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:03.642 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:03.659 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:03.659 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:03.679 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:03.679 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:03.699 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:03.699 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:03.722 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:03.721+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:03.723 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:03.723 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:03.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:03.823 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:03.820+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:03.902 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:03.902 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:03.917 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:03.923 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:03.921+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:03.956 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:03:04.025 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:04.025+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:04.074 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:04.074 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:04.091 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:04.138 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:04.136+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:04.226 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:04.226 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:04.226 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:04.225+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:04.242 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:04.327 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:04.325+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:04.422 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:04.422 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:04.427 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:04.424+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:04.437 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:04.529 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:04.529+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:04.607 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:04.607 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:04.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:04.666 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:04.664+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:04.731 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:04.728+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:04.746 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:04.746 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:04.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:04.850 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:04.848+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:04.923 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:04.923 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:04.932 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:04.933+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:04.937 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:04.945 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:03:04.946 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:03:04.946 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-22T01:03:04.946 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-22T01:03:04.974 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:03:05.054 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:05.052+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:05.110 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:05.110 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:05.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:05.170 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:05.168+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:05.236 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:05.233+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:05.287 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:05.287 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:05.302 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:05.306 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-22T01:03:05.304+0000 7fa2a81e5640 -1 mon.a@0(leader).osd e139 definitely_dead 0 2024-09-22T01:03:05.336 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:05.333+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:05.428 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:05.428 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:05.436 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:05.432+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:05.442 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:05.537 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:05.532+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:05.606 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:05.607 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:05.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:05.637 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:05.637+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:05.718 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:05.738 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:05.737+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:05.795 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:05.795 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:05.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:05.839 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:05.837+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:05.866 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:05.866 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:05.883 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:05.883 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:05.900 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:05.900 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:05.917 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:05.918 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:05.934 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:05.934 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:05.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:05.935 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:05.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.935 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54906 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:05.935 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:03:05.935 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458900084000007, 2024-09-22T01:03:05.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066071787000000007, 2024-09-22T01:03:05.936 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:05.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:05.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:05.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:05.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:05.937 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:05.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54906 2024-09-22T01:03:05.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.937 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:05.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:05.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:03:05.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:05.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:03:05.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:05.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473631+0000", 2024-09-22T01:03:05.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-22T01:03:05.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:05.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473633+0000", 2024-09-22T01:03:05.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.272e-06 2024-09-22T01:03:05.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:05.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473640+0000", 2024-09-22T01:03:05.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:03:05.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:05.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473645+0000", 2024-09-22T01:03:05.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1359999999999996e-06 2024-09-22T01:03:05.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:05.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539387+0000", 2024-09-22T01:03:05.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065742272000000004 2024-09-22T01:03:05.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:05.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539469+0000", 2024-09-22T01:03:05.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1465999999999996e-05 2024-09-22T01:03:05.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:05.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539702+0000", 2024-09-22T01:03:05.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023359499999999999 2024-09-22T01:03:05.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.945 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:05.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.946 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54907 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:05.946 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:03:05.946 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458879254999999, 2024-09-22T01:03:05.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066386601000000003, 2024-09-22T01:03:05.947 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:05.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:05.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:05.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:05.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:05.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:05.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54907 2024-09-22T01:03:05.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.948 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:05.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:05.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:03:05.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:05.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:03:05.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:05.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473652+0000", 2024-09-22T01:03:05.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:03:05.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:05.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473657+0000", 2024-09-22T01:03:05.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4779999999999999e-06 2024-09-22T01:03:05.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:05.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473664+0000", 2024-09-22T01:03:05.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-22T01:03:05.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:05.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473671+0000", 2024-09-22T01:03:05.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.126e-06 2024-09-22T01:03:05.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:05.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539734+0000", 2024-09-22T01:03:05.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066062885000000002 2024-09-22T01:03:05.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:05.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.539814+0000", 2024-09-22T01:03:05.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0167999999999995e-05 2024-09-22T01:03:05.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:05.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540038+0000", 2024-09-22T01:03:05.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022368600000000001 2024-09-22T01:03:05.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:05.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.956 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54908 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 143360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:05.956 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:03:05.957 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458850740000003, 2024-09-22T01:03:05.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066747865000000003, 2024-09-22T01:03:05.957 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:05.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:05.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:05.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:05.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:05.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:05.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54908 2024-09-22T01:03:05.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.958 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:05.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:05.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:03:05.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:05.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:03:05.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:05.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473680+0000", 2024-09-22T01:03:05.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:03:05.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:05.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473683+0000", 2024-09-22T01:03:05.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:03:05.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:05.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473691+0000", 2024-09-22T01:03:05.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8290000000000008e-06 2024-09-22T01:03:05.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:05.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473696+0000", 2024-09-22T01:03:05.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.074e-06 2024-09-22T01:03:05.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:05.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540069+0000", 2024-09-22T01:03:05.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066373504999999999 2024-09-22T01:03:05.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:05.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540149+0000", 2024-09-22T01:03:05.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0112999999999997e-05 2024-09-22T01:03:05.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:05.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540428+0000", 2024-09-22T01:03:05.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027859899999999998 2024-09-22T01:03:05.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.966 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:05.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54909 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:05.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:03:05.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458828089999997, 2024-09-22T01:03:05.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067053732000000005, 2024-09-22T01:03:05.972 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:05.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:05.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:05.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:05.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:05.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:05.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54909 2024-09-22T01:03:05.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.973 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:05.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:05.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:03:05.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:05.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:03:05.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:05.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473703+0000", 2024-09-22T01:03:05.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-22T01:03:05.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:05.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473705+0000", 2024-09-22T01:03:05.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9740000000000001e-06 2024-09-22T01:03:05.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:05.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473713+0000", 2024-09-22T01:03:05.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9249999999999995e-06 2024-09-22T01:03:05.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:05.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473718+0000", 2024-09-22T01:03:05.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.933e-06 2024-09-22T01:03:05.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:05.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540459+0000", 2024-09-22T01:03:05.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066741274000000003 2024-09-22T01:03:05.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:05.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540540+0000", 2024-09-22T01:03:05.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0474000000000002e-05 2024-09-22T01:03:05.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:05.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540756+0000", 2024-09-22T01:03:05.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021671599999999999 2024-09-22T01:03:05.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.981 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:05.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.982 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54910 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:05.982 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:03:05.982 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458806095, 2024-09-22T01:03:05.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067366913, 2024-09-22T01:03:05.982 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:05.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:05.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:05.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:05.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:05.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:05.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54910 2024-09-22T01:03:05.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:05.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:05.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:03:05.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:05.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:03:05.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:05.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473725+0000", 2024-09-22T01:03:05.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7100000000000002e-07 2024-09-22T01:03:05.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:05.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473727+0000", 2024-09-22T01:03:05.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-22T01:03:05.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:05.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473738+0000", 2024-09-22T01:03:05.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0288999999999999e-05 2024-09-22T01:03:05.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:05.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473743+0000", 2024-09-22T01:03:05.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1789999999999999e-06 2024-09-22T01:03:05.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:05.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540788+0000", 2024-09-22T01:03:05.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067045100999999996 2024-09-22T01:03:05.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:05.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.540868+0000", 2024-09-22T01:03:05.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0007999999999997e-05 2024-09-22T01:03:05.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:05.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541092+0000", 2024-09-22T01:03:05.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022375800000000001 2024-09-22T01:03:05.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:05.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54911 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458781031000001, 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067570937999999997, 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54911 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473750+0000", 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473752+0000", 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473760+0000", 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4510000000000003e-06 2024-09-22T01:03:05.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473764+0000", 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8049999999999997e-06 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541126+0000", 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067361587000000001 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541173+0000", 2024-09-22T01:03:05.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7394000000000001e-05 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541321+0000", 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014720999999999999 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54912 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458758598000003, 2024-09-22T01:03:05.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067737169999999999, 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54912 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:05.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473772+0000", 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473773+0000", 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.75e-07 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:05.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473775+0000", 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2560000000000001e-06 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473782+0000", 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.968e-06 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473787+0000", 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6809999999999997e-06 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541339+0000", 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067552833000000007 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541381+0000", 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.189e-05 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541509+0000", 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012806699999999999 2024-09-22T01:03:06.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54913 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458737120999999, 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067920799000000004, 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54913 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473794+0000", 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473796+0000", 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473803+0000", 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:03:06.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473814+0000", 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1146e-05 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541527+0000", 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067712957000000004 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541568+0000", 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1121000000000002e-05 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541714+0000", 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014606299999999999 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54914 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:03:06.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.458708363, 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068083414999999994, 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54914 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:03:06.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.473822+0000", 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474590+0000", 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076809299999999997 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474595+0000", 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474607+0000", 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.172e-05 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474622+0000", 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5124e-05 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541732+0000", 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067110239000000002 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541775+0000", 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2731999999999997e-05 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541906+0000", 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013075700000000001 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54915 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.457895371000006, 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067467545000000004, 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54915 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474635+0000", 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474636+0000", 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474639+0000", 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474647+0000", 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474652+0000", 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-06 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541924+0000", 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067272230000000002 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.541966+0000", 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1118999999999997e-05 2024-09-22T01:03:06.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542103+0000", 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137317 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54916 3.8 3:15ac7a96:::rbd_data.117cd666b4d1.0000000000000002:1b [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.457870686000007, 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067718964000000006, 2024-09-22T01:03:06.015 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54916 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.474660+0000", 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478699+0000", 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040384979999999997 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478711+0000", 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478727+0000", 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478735+0000", 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8620000000000003e-06 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542121+0000", 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063386473999999998 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542162+0000", 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1216e-05 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.542379+0000", 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216639 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54982 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453549151999994, 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065834138, 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54982 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478982+0000", 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478990+0000", 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9060000000000002e-06 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.478998+0000", 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6279999999999996e-06 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479003+0000", 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544696+0000", 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065693496000000004 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544736+0000", 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9737999999999998e-05 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544816+0000", 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9857999999999993e-05 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54983 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453520300999998, 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065957247999999996, 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54983 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479010+0000", 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479011+0000", 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-22T01:03:06.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479013+0000", 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479020+0000", 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7479999999999996e-06 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479025+0000", 2024-09-22T01:03:06.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0629999999999996e-06 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544833+0000", 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065807426000000002 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544887+0000", 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4567000000000001e-05 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544968+0000", 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0545999999999998e-05 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54984 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453498576000001, 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066072430000000001, 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54984 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479032+0000", 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479033+0000", 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479035+0000", 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479041+0000", 2024-09-22T01:03:06.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3589999999999996e-06 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479046+0000", 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8160000000000001e-06 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.544985+0000", 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065938605999999997 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545024+0000", 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9789999999999997e-05 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545105+0000", 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0128000000000003e-05 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54985 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:03:06.032 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453478145999995, 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066280326000000001, 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54985 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479053+0000", 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479055+0000", 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3130000000000001e-06 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479062+0000", 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1770000000000003e-06 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479068+0000", 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545156+0000", 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066088703999999998 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545237+0000", 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0813e-05 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545333+0000", 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5681999999999996e-05 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.036 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54986 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453455829999996, 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066407640000000004, 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54986 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479075+0000", 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479105+0000", 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9833999999999999e-05 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479116+0000", 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479133+0000", 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545362+0000", 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066228929000000006 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545402+0000", 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0395999999999997e-05 2024-09-22T01:03:06.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545483+0000", 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0634000000000001e-05 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54987 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 311296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453388035000003, 2024-09-22T01:03:06.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066473574999999993, 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54987 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:03:06.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479143+0000", 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0299999999999999e-07 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479146+0000", 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6350000000000002e-06 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479155+0000", 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6880000000000005e-06 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479168+0000", 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3008e-05 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545500+0000", 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066332279999999993 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545541+0000", 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1273999999999998e-05 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545616+0000", 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5186999999999997e-05 2024-09-22T01:03:06.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54988 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453354898000001, 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066778105000000004, 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54988 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479176+0000", 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9299999999999998e-07 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479179+0000", 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479187+0000", 2024-09-22T01:03:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479193+0000", 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3759999999999999e-06 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545671+0000", 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066478645000000003 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545859+0000", 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018741499999999999 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545954+0000", 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5295000000000002e-05 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54989 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 368640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453330976000004, 2024-09-22T01:03:06.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066957378999999997, 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54989 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479200+0000", 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479203+0000", 2024-09-22T01:03:06.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479211+0000", 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1969999999999996e-06 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479216+0000", 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8080000000000004e-06 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.545985+0000", 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066769414999999999 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546065+0000", 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0186999999999996e-05 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546157+0000", 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1947000000000006e-05 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.054 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:54990 3.8 3:12c27549:::rbd_data.117cd666b4d1.0000000000000006:1b [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.453307378999995, 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067085461999999998, 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.055 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54990 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.479223+0000", 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482492+0000", 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003268127 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482498+0000", 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1430000000000003e-06 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482512+0000", 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4527e-05 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.482520+0000", 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0020000000000006e-06 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546187+0000", 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063666608 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546228+0000", 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369999999999999e-05 2024-09-22T01:03:06.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.546309+0000", 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0685000000000004e-05 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:06.059 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:06.060 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:06.060 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:06.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:06.062 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:06.062 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:06.062 INFO:teuthology.orchestra.run.smithi002.stderr:marked in osd.4. 2024-09-22T01:03:06.062 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:06.063 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-22T01:03:06.064 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:06.065 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:06.061+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:06.161 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:06.161+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:06.196 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:06.196 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:06.196 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:06.196 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:06.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:06.262 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:06.260+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:06.363 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:06.360+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:06.374 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:06.374 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:06.374 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:06.374 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:06.374 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:06.388 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:06.463 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:06.461+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:06.513 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.543125894, 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:03:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.54307337100001, 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:03:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.54302104999999, 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:03:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542855276, 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542826848, 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:03:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:03:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542799039, 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:03:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542770934, 2024-09-22T01:03:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:03:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.54274339200001, 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.54271574400001, 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:03:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:03:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542688599, 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:03:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.54266063999999, 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542632825, 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.54260588, 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:03:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:03:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:03:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542578179, 2024-09-22T01:03:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:03:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542463755, 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:03:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542424397, 2024-09-22T01:03:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:03:06.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542394242, 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542364984, 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.588 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:03:06.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542337108, 2024-09-22T01:03:06.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:03:06.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:06.596 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.542307734, 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:03:06.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:06.601 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:06.601 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:06.602 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:06.602 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:06.601+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:06.703 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:06.700+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:06.772 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:06.772 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:06.772 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:06.772 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:06.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:06.804 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:06.800+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:06.905 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:06.900+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:06.908 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:06.908 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:06.908 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:06.908 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:06.908 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:06.923 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:07.006 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:07.005+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:07.013 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:03:07.054 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:07.054 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:07.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:07.054 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:07.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.054 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.054 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.695182461000002, 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:03:07.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.695145491999995, 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:03:07.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.695117222999997, 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:03:07.063 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.067 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.695079347000004, 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.071 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.695040757000001, 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.072 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.694968428999999, 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.076 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:07.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:03:07.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.694930563, 2024-09-22T01:03:07.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:03:07.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:03:07.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.694888324000004, 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:03:07.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:03:07.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:03:07.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.694833939000006, 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:03:07.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:03:07.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:03:07.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.694733174000007, 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.093 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:07.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:07.097 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.694646249000002, 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:07.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:03:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:03:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.101 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.694587924999993, 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.694134774999995, 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:03:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:03:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:03:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.693496288999995, 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:03:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:03:07.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.693208691999999, 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:03:07.114 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.118 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.692605723, 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:03:07.122 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.691525424000005, 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.123 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.691455546, 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:07.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:07.131 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.690893555000002, 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:07.135 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.690516130000006, 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:03:07.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:03:07.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.139 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:07.141 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:07.141 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:07.136+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:07.242 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:07.240+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:07.310 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:07.310 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:07.310 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:07.310 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:07.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:07.369 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:07.369+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:07.444 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:07.440+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:07.448 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:07.448 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:07.448 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:07.449 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:07.449 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:07.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:07.544 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:07.540+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 155.73916541200001, 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.633 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:03:07.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.355168085000003, 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.35513478, 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:03:07.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.355038266999998, 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.648 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354965015999994, 2024-09-22T01:03:07.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:07.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:07.656 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354899834999998, 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:07.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.660 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354829714000005, 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.664 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354796348999997, 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:03:07.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.668 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354765927000003, 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:03:07.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:03:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354735621000003, 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:03:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354705218999996, 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:03:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:03:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354674419999995, 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:03:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:03:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354643947, 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:03:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354613354999998, 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:03:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354575553000004, 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354543187999994, 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:03:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354512869000004, 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:03:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:03:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:03:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354482977000004, 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:03:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354453218000003, 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.354402753000002, 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:07.719 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:07.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:07.721 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:07.716+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:03:07.822 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:07.820+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:07.886 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:07.886 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:07.887 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:07.887 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:07.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:07.949 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:07.949+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:08.023 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:08.021+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:08.041 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:08.042 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:08.042 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:08.042 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:08.042 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:08.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:08.065 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:08.124 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:08.120+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:08.225 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:08.225+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:08.226 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:08.226 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:08.243 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:08.244 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:08.244 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.114308148000006, 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:08.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.114252472999993, 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:03:08.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.114200127999993, 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:03:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.114142216999994, 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:03:08.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:03:08.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.114083578000006, 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.270 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.114028345999998, 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.275 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.113974622000001, 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.113902886999995, 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:03:08.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:03:08.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.113845506000004, 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.286 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:03:08.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:03:08.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.113795240000002, 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:03:08.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:03:08.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.113716182999994, 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.296 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:08.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:03:08.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.113591933999999, 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.301 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:03:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:08.306 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.113511352000003, 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:03:08.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:03:08.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.311 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.113446163999996, 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:03:08.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.112671195000004, 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.317 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:03:08.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.112621410000003, 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.322 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:03:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:03:08.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:03:08.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.112540308999996, 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:03:08.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.331 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.112470324, 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:03:08.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:03:08.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:03:08.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.112392959999994, 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:03:08.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.112332438999999, 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:08.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:08.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:03:08.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:08.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:08.347 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:08.348 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:08.349 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:08.349 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:08.350 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:08.350 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:08.350 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:08.351 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:08.351 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:08.351 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:08.352 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:08.352 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:08.352 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:08.344+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:08.357 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:08.357 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:08.450 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:08.448+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:08.470 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:08.470 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:08.484 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:08.550 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:08.549+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:08.651 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:08.649+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:08.662 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:08.662 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:08.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:08.752 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:08.749+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:08.821 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:08.821 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:08.836 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:08.886 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:08.884+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:08.954 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:08.953+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:08.967 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:08.967 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:08.983 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:09.054 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:09.053+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:09.155 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:09.153+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:09.158 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:09.158 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:09.173 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:09.257 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:09.252+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:09.312 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:09.312 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:09.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:09.370 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:09.368+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:09.459 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:09.457+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:09.502 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:09.502 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:09.518 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:09.559 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:09.557+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:09.645 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:09.645 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:09.660 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:09.661 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:09.657+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:09.760 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:09.757+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:09.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:03:09.794 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:09.794 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:09.811 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:09.898 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:09.896+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:09.962 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:09.961+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:09.974 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:09.974 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:09.990 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:10.082 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:10.080+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:10.164 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:10.160+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:10.171 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:10.265 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:10.265+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:10.340 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:10.546 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:10.547 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:10.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:10.549 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:10.545+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:10.649 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:10.644+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:10.691 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:10.691 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:10.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:10.736 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:03:10.753 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:10.753+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:10.758 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:10.758 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:10.778 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:10.778 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:10.795 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:10.795 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:10.814 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:10.814 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:10.835 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:10.836 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:10.842 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:10.842 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:10.842 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:10.842 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:10.842 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:10.849 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:10.844+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:10.856 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:10.856 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:10.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:10.877 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:10.896 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:10.896 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:10.954 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:10.952+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:11.035 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:11.035 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:11.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:11.035 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:11.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.035 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.065193569, 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:03:11.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:03:11.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06514104599999, 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.065088725, 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:03:11.044 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:03:11.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:11.048 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.064922951, 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:11.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:03:11.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06489452300001, 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:03:11.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:03:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.064866714, 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.057 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:03:11.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06483860900001, 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.061 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:03:11.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06481106699999, 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.065 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:03:11.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06478341899999, 2024-09-22T01:03:11.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:11.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.073 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.064756274, 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:03:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.064728315, 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:03:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.0647005, 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.064673555, 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06464585400001, 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:03:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06453143, 2024-09-22T01:03:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:03:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06449207199999, 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:03:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.064461917, 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.064432659, 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:03:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.064404783, 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:03:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:03:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.06437540899999, 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:03:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.120 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:11.120 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:03:11.120 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:03:11.120 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-22T01:03:11.120 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:03:11.122 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:11.122 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:11.121+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:11.222 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:11.221+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:11.313 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:11.313 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:11.313 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:11.313 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:11.323 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:11.320+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:11.329 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:11.423 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:11.421+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 145, 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-22T01:00:18.015746+0000", 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-22T01:03:10.734939+0000", 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-22T01:01:26.960327+0000", 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-22T01:03:05.961739+0000", 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:03:11.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5801984, 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "noscrub", 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-22T01:03:11.449 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:30.699559+0000", 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-22T01:03:11.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-22T01:03:11.451 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:11.452 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:11.453 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:11.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-22T01:03:11.455 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:11.456 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:36.410605+0000", 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:11.457 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:11.458 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 2, 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 21, 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:11.459 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:11.460 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:11.461 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9900000095367432, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9900000095367432, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:03:11.462 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:46.342370+0000", 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "143", 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 50, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 143, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:11.466 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:11.468 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.75, 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.75, 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.469 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "8c0f94a8-3c63-442d-9dc1-50c1278929da", 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 144, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:11.470 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.471 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.472 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2277223948", 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2277223948", 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2277223948", 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2277223948", 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "2d2cf30e-e9bd-40ba-8293-193faa02ea69", 2024-09-22T01:03:11.475 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 142, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.476 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-22T01:03:11.477 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.478 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:11.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/463340194", 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/463340194", 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/463340194", 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/463340194", 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:11.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a3d3e330-9ea1-4a5c-811b-f285f9a3cff4", 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 143, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:11.481 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-22T01:03:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/387643397", 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/387643397", 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/387643397", 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/387643397", 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "239072f0-4b69-4624-ba6c-733f299208da", 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 144, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1215210764", 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1215210764", 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1215210764", 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1215210764", 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fd35cb24-eb2c-4c17-abe9-ce5c34db9014", 2024-09-22T01:03:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 144, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6800", 2024-09-22T01:03:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6801", 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6802", 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6803", 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6806", 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6807", 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6804", 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6805", 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6801/2463696152", 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6803/2463696152", 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6807/2463696152", 2024-09-22T01:03:11.497 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6805/2463696152", 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "b60a584d-5334-41e4-92d9-f6689de6b8bf", 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:11.498 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 15, 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 25, 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 52, 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 144, 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 26, 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6824", 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6825", 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6826", 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6827", 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6830", 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6831", 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6828", 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:11.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6829", 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6825/1737475131", 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6827/1737475131", 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1737475131", 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1737475131", 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:11.503 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "9be24894-1100-4e4e-bf1b-79e163fd97e0", 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 144, 2024-09-22T01:03:11.504 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6816", 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6817", 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6818", 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6819", 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6822", 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6823", 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6820", 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6821", 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6817/4220217922", 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6819/4220217922", 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6823/4220217922", 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6821/4220217922", 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:03:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "293cfbc3-8c2e-40b5-ab89-0a124cc8651c", 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 144, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6808", 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6809", 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:11.511 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6810", 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6811", 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:11.512 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6814", 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6815", 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.513 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6812", 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6813", 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6809/3867156106", 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6811/3867156106", 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3867156106", 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3867156106", 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:03:11.515 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.556128+0000", 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:11.516 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.159753+0000", 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.766199+0000", 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:11.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.155529+0000", 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:03:11.518 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.226232+0000", 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:11.519 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.243346+0000", 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 26 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.143084+0000", 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.524369+0000", 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.11", 2024-09-22T01:03:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: 4, 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: 6 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.17", 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: 1 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [ 2024-09-22T01:03:11.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.13", 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 7, 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 3 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [ 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.523 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.3", 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: 0 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.a", 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: 6 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.1b", 2024-09-22T01:03:11.524 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: 0 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-22T01:03:11.525 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [ 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 29, 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 6 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:11.526 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:11.527 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:11.529 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:11.529 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:11.529 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:11.529 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:11.529 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:11.530 INFO:tasks.thrashosds.thrasher:j is {'epoch': 145, 'fsid': '7d08b428-4185-4f0f-a0fc-f7dc1fefdce9', 'created': '2024-09-22T01:00:18.015746+0000', 'modified': '2024-09-22T01:03:10.734939+0000', 'last_up_change': '2024-09-22T01:01:26.960327+0000', 'last_in_change': '2024-09-22T01:03:05.961739+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:00:30.699559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:00:36.410605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:00:46.342370+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 143, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8c0f94a8-3c63-442d-9dc1-50c1278929da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2277223948}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2277223948}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2277223948}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2277223948}]}, 'public_addr': '172.21.15.2:6825/2277223948', 'cluster_addr': '172.21.15.2:6827/2277223948', 'heartbeat_back_addr': '172.21.15.2:6831/2277223948', 'heartbeat_front_addr': '172.21.15.2:6829/2277223948', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 463340194}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 463340194}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 463340194}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 463340194}]}, 'public_addr': '172.21.15.2:6817/463340194', 'cluster_addr': '172.21.15.2:6819/463340194', 'heartbeat_back_addr': '172.21.15.2:6823/463340194', 'heartbeat_front_addr': '172.21.15.2:6821/463340194', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 387643397}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 387643397}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 387643397}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 387643397}]}, 'public_addr': '172.21.15.2:6801/387643397', 'cluster_addr': '172.21.15.2:6803/387643397', 'heartbeat_back_addr': '172.21.15.2:6807/387643397', 'heartbeat_front_addr': '172.21.15.2:6805/387643397', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '239072f0-4b69-4624-ba6c-733f299208da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1215210764}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1215210764}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1215210764}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1215210764}]}, 'public_addr': '172.21.15.2:6809/1215210764', 'cluster_addr': '172.21.15.2:6811/1215210764', 'heartbeat_back_addr': '172.21.15.2:6815/1215210764', 'heartbeat_front_addr': '172.21.15.2:6813/1215210764', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 2463696152}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 2463696152}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 2463696152}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 2463696152}]}, 'public_addr': '172.21.15.119:6801/2463696152', 'cluster_addr': '172.21.15.119:6803/2463696152', 'heartbeat_back_addr': '172.21.15.119:6807/2463696152', 'heartbeat_front_addr': '172.21.15.119:6805/2463696152', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b60a584d-5334-41e4-92d9-f6689de6b8bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 25, 'up_from': 52, 'up_thru': 144, 'down_at': 26, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1737475131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1737475131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1737475131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1737475131}]}, 'public_addr': '172.21.15.119:6825/1737475131', 'cluster_addr': '172.21.15.119:6827/1737475131', 'heartbeat_back_addr': '172.21.15.119:6831/1737475131', 'heartbeat_front_addr': '172.21.15.119:6829/1737475131', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9be24894-1100-4e4e-bf1b-79e163fd97e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 4220217922}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 4220217922}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 4220217922}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 4220217922}]}, 'public_addr': '172.21.15.119:6817/4220217922', 'cluster_addr': '172.21.15.119:6819/4220217922', 'heartbeat_back_addr': '172.21.15.119:6823/4220217922', 'heartbeat_front_addr': '172.21.15.119:6821/4220217922', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '293cfbc3-8c2e-40b5-ab89-0a124cc8651c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3867156106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3867156106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3867156106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3867156106}]}, 'public_addr': '172.21.15.119:6809/3867156106', 'cluster_addr': '172.21.15.119:6811/3867156106', 'heartbeat_back_addr': '172.21.15.119:6815/3867156106', 'heartbeat_front_addr': '172.21.15.119:6813/3867156106', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.556128+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.159753+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.766199+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.155529+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.226232+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.243346+0000', 'dead_epoch': 26}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.143084+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.524369+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.11', 'osds': [4, 6]}, {'pgid': '3.17', 'osds': [6, 1]}], 'pg_upmap_items': [{'pgid': '3.13', 'mappings': [{'from': 7, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.3', 'osds': [0]}, {'pgid': '3.a', 'osds': [6]}, {'pgid': '3.1b', 'osds': [0]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 29, 'length': 6}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:03:11.530 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:03:11.531 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:11.524+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:11.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:11.629 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:11.629+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:11.696 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.337135708000005, 2024-09-22T01:03:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:03:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:03:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:03:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.337098738999998, 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:03:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:03:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.33707047, 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.337032593999993, 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336994004000005, 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:03:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:03:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336921676000003, 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:03:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336883810000003, 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:11.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:03:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336841570999994, 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:03:11.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:03:11.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:03:11.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:03:11.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:11.737 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336787185999995, 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:03:11.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:03:11.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:03:11.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:03:11.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336686420999996, 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.743 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:03:11.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336599496000005, 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:03:11.748 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:11.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:03:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:03:11.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:03:11.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.754 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.755 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:11.755 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336541171999997, 2024-09-22T01:03:11.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:03:11.755 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.759 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.336088021999998, 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:11.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:03:11.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:03:11.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:03:11.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.764 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.764 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.764 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.335449535999999, 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:03:11.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.335161939000002, 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.769 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:03:11.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:03:11.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:03:11.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.334558970000003, 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:03:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:03:11.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.333478670999995, 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:03:11.780 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:11.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:03:11.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.333408793000004, 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:03:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.790 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.790 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:11.790 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.332846802000006, 2024-09-22T01:03:11.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:03:11.790 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:03:11.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74380 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7184384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.332469376999995, 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097365221000000002, 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:11.795 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74380 2024-09-22T01:03:11.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.796 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:11.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:11.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:11.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361149+0000", 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361614+0000", 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046465900000000002 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:11.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361654+0000", 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9919000000000002e-05 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361675+0000", 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0937e-05 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361687+0000", 2024-09-22T01:03:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2405999999999999e-05 2024-09-22T01:03:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458327+0000", 2024-09-22T01:03:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096640304999999996 2024-09-22T01:03:11.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:11.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458398+0000", 2024-09-22T01:03:11.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0376000000000005e-05 2024-09-22T01:03:11.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:11.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:11.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:11.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458514+0000", 2024-09-22T01:03:11.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116619 2024-09-22T01:03:11.801 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.801 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.801 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.801 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:11.801 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:11.801 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:11.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:11.803 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:11.800+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:03:11.886 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:03:11.903 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:11.901+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:11.905 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-22T01:03:11.739050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":609689915,"num_objects":1020,"num_object_clones":271,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":488,"num_read":196301,"num_read_kb":754520,"num_write":21662,"num_write_kb":413225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":204797527,"num_keys_recovered":904,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18277,"ondisk_log_size":18277,"up":82,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":537332,"kb_used_data":179836,"kb_used_omap":15,"kb_used_meta":357296,"kb_avail":754437388,"statfs":{"total":773094113280,"available":772543885312,"internally_reserved":0,"allocated":184152064,"data_stored":206339376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":365871801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15830234,"num_objects":-69,"num_object_clones":-30,"num_object_copies":-138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-69,"num_whiteouts":-23,"num_read":390,"num_read_kb":339,"num_write":73,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1133,"num_bytes_recovered":22767502,"num_keys_recovered":120,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.006529"},"pg_stats":[{"pgid":"3.1f","version":"144'493","reported_seq":6922,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.740323+0000","last_change":"2024-09-22T01:03:08.760680+0000","last_active":"2024-09-22T01:03:09.740323+0000","last_peered":"2024-09-22T01:03:09.740323+0000","last_clean":"2024-09-22T01:03:09.740323+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:03:09.740323+0000","last_undegraded":"2024-09-22T01:03:09.740323+0000","last_fullsized":"2024-09-22T01:03:09.740323+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:45:43.769870+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.023381637,"stat_sum":{"num_bytes":26189761,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":6502,"num_read_kb":45938,"num_write":531,"num_write_kb":17793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.1e","version":"144'712","reported_seq":3328,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.864212+0000","last_change":"2024-09-22T01:03:08.765246+0000","last_active":"2024-09-22T01:03:09.864212+0000","last_peered":"2024-09-22T01:03:09.864212+0000","last_clean":"2024-09-22T01:03:09.864212+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:03:09.864212+0000","last_undegraded":"2024-09-22T01:03:09.864212+0000","last_fullsized":"2024-09-22T01:03:09.864212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:32.830835+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.012484299000000001,"stat_sum":{"num_bytes":22108336,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":6084,"num_read_kb":12757,"num_write":934,"num_write_kb":7191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.1d","version":"144'451","reported_seq":4956,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.743091+0000","last_change":"2024-09-22T01:03:09.769202+0000","last_active":"2024-09-22T01:03:10.743091+0000","last_peered":"2024-09-22T01:03:10.743091+0000","last_clean":"2024-09-22T01:03:10.743091+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:03:10.743091+0000","last_undegraded":"2024-09-22T01:03:10.743091+0000","last_fullsized":"2024-09-22T01:03:10.743091+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:09.672732+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084803589999999998,"stat_sum":{"num_bytes":22072279,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":4787,"num_read_kb":32328,"num_write":474,"num_write_kb":18066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.1c","version":"144'1046","reported_seq":10280,"reported_epoch":144,"state":"peering","last_fresh":"2024-09-22T01:03:10.348086+0000","last_change":"2024-09-22T01:03:10.743696+0000","last_active":"2024-09-22T01:03:10.743696+0000","last_peered":"2024-09-22T01:03:10.348086+0000","last_clean":"2024-09-22T01:03:05.093436+0000","last_became_active":"2024-09-22T01:03:07.797891+0000","last_became_peered":"2024-09-22T01:03:07.797891+0000","last_unstale":"2024-09-22T01:03:10.348086+0000","last_undegraded":"2024-09-22T01:03:10.348086+0000","last_fullsized":"2024-09-22T01:03:07.740370+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.016616188,"stat_sum":{"num_bytes":43569411,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":11313,"num_read_kb":51555,"num_write":1533,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"4"}]},{"pgid":"3.1b","version":"142'1054","reported_seq":2619,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T01:03:09.740118+0000","last_change":"2024-09-22T01:03:07.803754+0000","last_active":"2024-09-22T01:03:09.740118+0000","last_peered":"2024-09-22T01:03:09.740118+0000","last_clean":"2024-09-22T01:03:05.197600+0000","last_became_active":"2024-09-22T01:03:07.802786+0000","last_became_peered":"2024-09-22T01:03:07.802786+0000","last_unstale":"2024-09-22T01:03:09.740118+0000","last_undegraded":"2024-09-22T01:03:07.751423+0000","last_fullsized":"2024-09-22T01:03:07.747612+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1054,"log_dups_size":0,"ondisk_log_size":1054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011003209999999999,"stat_sum":{"num_bytes":16309537,"num_objects":38,"num_object_clones":16,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":113,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":5192,"num_read_kb":15408,"num_write":1511,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":100534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","7"],"object_location_counts":[{"shards":"0","objects":-1},{"shards":"0,7","objects":38}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"4"}]},{"pgid":"3.1a","version":"139'466","reported_seq":11947,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.854816+0000","last_change":"2024-09-22T01:03:09.760288+0000","last_active":"2024-09-22T01:03:09.854816+0000","last_peered":"2024-09-22T01:03:09.854816+0000","last_clean":"2024-09-22T01:03:09.854816+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:03:09.854816+0000","last_undegraded":"2024-09-22T01:03:09.854816+0000","last_fullsized":"2024-09-22T01:03:09.854816+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:09.986222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030427789999999998,"stat_sum":{"num_bytes":20344556,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":14,"num_read":11982,"num_read_kb":25824,"num_write":538,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.19","version":"139'375","reported_seq":9820,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.741233+0000","last_change":"2024-09-22T01:03:08.738905+0000","last_active":"2024-09-22T01:03:09.741233+0000","last_peered":"2024-09-22T01:03:09.741233+0000","last_clean":"2024-09-22T01:03:09.741233+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:03:09.741233+0000","last_undegraded":"2024-09-22T01:03:09.741233+0000","last_fullsized":"2024-09-22T01:03:09.741233+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:26:58.305875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001480313,"stat_sum":{"num_bytes":22690867,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":9122,"num_read_kb":31298,"num_write":444,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.18","version":"143'717","reported_seq":6239,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.741562+0000","last_change":"2024-09-22T01:03:08.758639+0000","last_active":"2024-09-22T01:03:09.741562+0000","last_peered":"2024-09-22T01:03:09.741562+0000","last_clean":"2024-09-22T01:03:09.741562+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:03:09.741562+0000","last_undegraded":"2024-09-22T01:03:09.741562+0000","last_fullsized":"2024-09-22T01:03:09.741562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:00:10.566519+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0061021590000000002,"stat_sum":{"num_bytes":39041357,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":8356,"num_read_kb":42508,"num_write":803,"num_write_kb":16840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.17","version":"143'514","reported_seq":4040,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.742326+0000","last_change":"2024-09-22T01:03:08.764193+0000","last_active":"2024-09-22T01:03:10.742326+0000","last_peered":"2024-09-22T01:03:10.742326+0000","last_clean":"2024-09-22T01:03:10.742326+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:03:10.742326+0000","last_undegraded":"2024-09-22T01:03:10.742326+0000","last_fullsized":"2024-09-22T01:03:10.742326+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:52:54.929216+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.020389655999999999,"stat_sum":{"num_bytes":9973575,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":4686,"num_read_kb":18204,"num_write":597,"num_write_kb":8516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.16","version":"144'346","reported_seq":2017,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.744163+0000","last_change":"2024-09-22T01:03:08.755461+0000","last_active":"2024-09-22T01:03:10.744163+0000","last_peered":"2024-09-22T01:03:10.744163+0000","last_clean":"2024-09-22T01:03:10.744163+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:03:10.744163+0000","last_undegraded":"2024-09-22T01:03:10.744163+0000","last_fullsized":"2024-09-22T01:03:10.744163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:22.360998+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010305119999999999,"stat_sum":{"num_bytes":12962008,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":2577,"num_read_kb":34283,"num_write":384,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.15","version":"143'600","reported_seq":6389,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.744121+0000","last_change":"2024-09-22T01:03:08.767355+0000","last_active":"2024-09-22T01:03:09.744121+0000","last_peered":"2024-09-22T01:03:09.744121+0000","last_clean":"2024-09-22T01:03:09.744121+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:03:09.744121+0000","last_undegraded":"2024-09-22T01:03:09.744121+0000","last_fullsized":"2024-09-22T01:03:09.744121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:56:55.011652+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012955393000000001,"stat_sum":{"num_bytes":19857350,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":8435,"num_read_kb":26741,"num_write":634,"num_write_kb":5749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.14","version":"139'490","reported_seq":13563,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-22T01:03:04.987114+0000","last_change":"2024-09-22T01:03:04.987978+0000","last_active":"2024-09-22T01:03:04.987114+0000","last_peered":"2024-09-22T01:03:04.987114+0000","last_clean":"2024-09-22T01:03:04.987114+0000","last_became_active":"2024-09-22T01:01:43.138527+0000","last_became_peered":"2024-09-22T01:01:43.138527+0000","last_unstale":"2024-09-22T01:03:04.987114+0000","last_undegraded":"2024-09-22T01:03:04.987114+0000","last_fullsized":"2024-09-22T01:03:04.987114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:12:26.753116+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.019356563,"stat_sum":{"num_bytes":27507875,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":13638,"num_read_kb":40761,"num_write":516,"num_write_kb":17919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1907181,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.742482+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:03:10.742482+0000","last_peered":"2024-09-22T01:03:10.742482+0000","last_clean":"2024-09-22T01:03:10.742482+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:03:10.742482+0000","last_undegraded":"2024-09-22T01:03:10.742482+0000","last_fullsized":"2024-09-22T01:03:10.742482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"143'499","reported_seq":2724,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.740416+0000","last_change":"2024-09-22T01:03:08.740302+0000","last_active":"2024-09-22T01:03:09.740416+0000","last_peered":"2024-09-22T01:03:09.740416+0000","last_clean":"2024-09-22T01:03:09.740416+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:03:09.740416+0000","last_undegraded":"2024-09-22T01:03:09.740416+0000","last_fullsized":"2024-09-22T01:03:09.740416+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:49:19.251549+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.004694223,"stat_sum":{"num_bytes":15562799,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":15,"num_read":3683,"num_read_kb":6161,"num_write":502,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":80,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.741010+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:03:09.741010+0000","last_peered":"2024-09-22T01:03:09.741010+0000","last_clean":"2024-09-22T01:03:09.741010+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:03:09.741010+0000","last_undegraded":"2024-09-22T01:03:09.741010+0000","last_fullsized":"2024-09-22T01:03:09.741010+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'537","reported_seq":3383,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.741759+0000","last_change":"2024-09-22T01:03:08.765391+0000","last_active":"2024-09-22T01:03:09.741759+0000","last_peered":"2024-09-22T01:03:09.741759+0000","last_clean":"2024-09-22T01:03:09.741759+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:03:09.741759+0000","last_undegraded":"2024-09-22T01:03:09.741759+0000","last_fullsized":"2024-09-22T01:03:09.741759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:36:08.848951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012478098999999999,"stat_sum":{"num_bytes":13547440,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":15,"num_read":4463,"num_read_kb":12695,"num_write":674,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.5","version":"0'0","reported_seq":266,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.744654+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:03:09.744654+0000","last_peered":"2024-09-22T01:03:09.744654+0000","last_clean":"2024-09-22T01:03:09.744654+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:03:09.744654+0000","last_undegraded":"2024-09-22T01:03:09.744654+0000","last_fullsized":"2024-09-22T01:03:09.744654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"143'399","reported_seq":4153,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.740896+0000","last_change":"2024-09-22T01:03:08.742714+0000","last_active":"2024-09-22T01:03:09.740896+0000","last_peered":"2024-09-22T01:03:09.740896+0000","last_clean":"2024-09-22T01:03:09.740896+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:03:09.740896+0000","last_undegraded":"2024-09-22T01:03:09.740896+0000","last_fullsized":"2024-09-22T01:03:09.740896+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:40:09.383110+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0046747020000000002,"stat_sum":{"num_bytes":9458418,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":3800,"num_read_kb":26304,"num_write":388,"num_write_kb":16759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":80,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.740631+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:03:09.740631+0000","last_peered":"2024-09-22T01:03:09.740631+0000","last_clean":"2024-09-22T01:03:09.740631+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:03:09.740631+0000","last_undegraded":"2024-09-22T01:03:09.740631+0000","last_fullsized":"2024-09-22T01:03:09.740631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"144'575","reported_seq":3992,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.743161+0000","last_change":"2024-09-22T01:03:08.762657+0000","last_active":"2024-09-22T01:03:10.743161+0000","last_peered":"2024-09-22T01:03:10.743161+0000","last_clean":"2024-09-22T01:03:10.743161+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:03:10.743161+0000","last_undegraded":"2024-09-22T01:03:10.743161+0000","last_fullsized":"2024-09-22T01:03:10.743161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:30:17.760293+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.01698473,"stat_sum":{"num_bytes":12592440,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":6331,"num_read_kb":12165,"num_write":514,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.3","version":"144'588","reported_seq":2525,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T01:03:09.843574+0000","last_change":"2024-09-22T01:03:07.801512+0000","last_active":"2024-09-22T01:03:09.843574+0000","last_peered":"2024-09-22T01:03:09.843574+0000","last_clean":"2024-09-22T01:03:05.150156+0000","last_became_active":"2024-09-22T01:03:07.799915+0000","last_became_peered":"2024-09-22T01:03:07.799915+0000","last_unstale":"2024-09-22T01:03:09.843574+0000","last_undegraded":"2024-09-22T01:03:07.746763+0000","last_fullsized":"2024-09-22T01:03:07.741442+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015089754,"stat_sum":{"num_bytes":11492499,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":3198,"num_read_kb":17820,"num_write":624,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3201594,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[0],"avail_no_missing":["0","6"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,6","objects":32}],"blocked_by":[],"up_primary":4,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"4"}]},{"pgid":"2.2","version":"21'2","reported_seq":240,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.742947+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:03:10.742947+0000","last_peered":"2024-09-22T01:03:10.742947+0000","last_clean":"2024-09-22T01:03:10.742947+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:03:10.742947+0000","last_undegraded":"2024-09-22T01:03:10.742947+0000","last_fullsized":"2024-09-22T01:03:10.742947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"143'1594","reported_seq":3572,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.740576+0000","last_change":"2024-09-22T01:03:08.742407+0000","last_active":"2024-09-22T01:03:09.740576+0000","last_peered":"2024-09-22T01:03:09.740576+0000","last_clean":"2024-09-22T01:03:09.740576+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:03:09.740576+0000","last_undegraded":"2024-09-22T01:03:09.740576+0000","last_fullsized":"2024-09-22T01:03:09.740576+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1594,"log_dups_size":0,"ondisk_log_size":1594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:39:44.974723+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047573700000000003,"stat_sum":{"num_bytes":19268696,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":8901,"num_read_kb":25309,"num_write":2663,"num_write_kb":16901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":266,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.738537+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:03:09.738537+0000","last_peered":"2024-09-22T01:03:09.738537+0000","last_clean":"2024-09-22T01:03:09.738537+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:03:09.738537+0000","last_undegraded":"2024-09-22T01:03:09.738537+0000","last_fullsized":"2024-09-22T01:03:09.738537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"139'354","reported_seq":2877,"reported_epoch":143,"state":"peering","last_fresh":"2024-09-22T01:03:09.254703+0000","last_change":"2024-09-22T01:03:09.740602+0000","last_active":"2024-09-22T01:03:09.740602+0000","last_peered":"2024-09-22T01:03:09.254703+0000","last_clean":"2024-09-22T01:03:04.980799+0000","last_became_active":"2024-09-22T01:03:07.797570+0000","last_became_peered":"2024-09-22T01:03:07.797570+0000","last_unstale":"2024-09-22T01:03:09.254703+0000","last_undegraded":"2024-09-22T01:03:09.254703+0000","last_fullsized":"2024-09-22T01:03:07.741095+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0082121889999999999,"stat_sum":{"num_bytes":9823319,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":2187,"num_read_kb":18579,"num_write":378,"num_write_kb":15217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":276,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.743679+0000","last_change":"2024-09-22T01:02:35.049954+0000","last_active":"2024-09-22T01:03:09.743679+0000","last_peered":"2024-09-22T01:03:09.743679+0000","last_clean":"2024-09-22T01:03:09.743679+0000","last_became_active":"2024-09-22T01:02:35.049024+0000","last_became_peered":"2024-09-22T01:02:35.049024+0000","last_unstale":"2024-09-22T01:03:09.743679+0000","last_undegraded":"2024-09-22T01:03:09.743679+0000","last_fullsized":"2024-09-22T01:03:09.743679+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:45.992332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"143'350","reported_seq":2113,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.741826+0000","last_change":"2024-09-22T01:03:08.764789+0000","last_active":"2024-09-22T01:03:09.741826+0000","last_peered":"2024-09-22T01:03:09.741826+0000","last_clean":"2024-09-22T01:03:09.741826+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:03:09.741826+0000","last_undegraded":"2024-09-22T01:03:09.741826+0000","last_fullsized":"2024-09-22T01:03:09.741826+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:17:51.016876+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011942328,"stat_sum":{"num_bytes":14954951,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":1621,"num_read_kb":15577,"num_write":354,"num_write_kb":17862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.3","version":"19'1","reported_seq":221,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.743170+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:03:10.743170+0000","last_peered":"2024-09-22T01:03:10.743170+0000","last_clean":"2024-09-22T01:03:10.743170+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:03:10.743170+0000","last_undegraded":"2024-09-22T01:03:10.743170+0000","last_fullsized":"2024-09-22T01:03:10.743170+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":417,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.743705+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:03:09.743705+0000","last_peered":"2024-09-22T01:03:09.743705+0000","last_clean":"2024-09-22T01:03:09.743705+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:03:09.743705+0000","last_undegraded":"2024-09-22T01:03:09.743705+0000","last_fullsized":"2024-09-22T01:03:09.743705+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"143'418","reported_seq":10322,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.741744+0000","last_change":"2024-09-22T01:03:08.758966+0000","last_active":"2024-09-22T01:03:09.741744+0000","last_peered":"2024-09-22T01:03:09.741744+0000","last_clean":"2024-09-22T01:03:09.741744+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:03:09.741744+0000","last_undegraded":"2024-09-22T01:03:09.741744+0000","last_fullsized":"2024-09-22T01:03:09.741744+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:53:03.115479+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0063836400000000003,"stat_sum":{"num_bytes":18773223,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":10548,"num_read_kb":27261,"num_write":411,"num_write_kb":13697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.9","version":"139'553","reported_seq":2208,"reported_epoch":144,"state":"peering","last_fresh":"2024-09-22T01:03:08.939677+0000","last_change":"2024-09-22T01:03:10.680418+0000","last_active":"2024-09-22T01:03:10.680418+0000","last_peered":"2024-09-22T01:03:08.939677+0000","last_clean":"2024-09-22T01:03:04.976257+0000","last_became_active":"2024-09-22T01:03:07.793264+0000","last_became_peered":"2024-09-22T01:03:07.793264+0000","last_unstale":"2024-09-22T01:03:08.939677+0000","last_undegraded":"2024-09-22T01:03:08.939677+0000","last_fullsized":"2024-09-22T01:03:07.739602+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.012465594,"stat_sum":{"num_bytes":21573681,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":2809,"num_read_kb":7150,"num_write":631,"num_write_kb":8025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"4"}]},{"pgid":"3.a","version":"145'728","reported_seq":15813,"reported_epoch":145,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:03:11.112014+0000","last_change":"2024-09-22T01:03:11.112258+0000","last_active":"2024-09-22T01:03:11.112014+0000","last_peered":"2024-09-22T01:03:11.112014+0000","last_clean":"2024-09-22T01:03:05.127768+0000","last_became_active":"2024-09-22T01:03:07.797797+0000","last_became_peered":"2024-09-22T01:03:07.797797+0000","last_unstale":"2024-09-22T01:03:11.112014+0000","last_undegraded":"2024-09-22T01:03:07.741939+0000","last_fullsized":"2024-09-22T01:03:07.735906+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.025425696000000001,"stat_sum":{"num_bytes":13395824,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":16382,"num_read_kb":40357,"num_write":766,"num_write_kb":13031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":13815709,"num_keys_recovered":200,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6],"avail_no_missing":["6","1","4"],"object_location_counts":[{"shards":"6","objects":31}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1f","length":"4"}]},{"pgid":"3.b","version":"139'614","reported_seq":2472,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.742891+0000","last_change":"2024-09-22T01:03:08.759814+0000","last_active":"2024-09-22T01:03:09.742891+0000","last_peered":"2024-09-22T01:03:09.742891+0000","last_clean":"2024-09-22T01:03:09.742891+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:03:09.742891+0000","last_undegraded":"2024-09-22T01:03:09.742891+0000","last_fullsized":"2024-09-22T01:03:09.742891+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:14:06.983814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070549810000000001,"stat_sum":{"num_bytes":6978235,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":4007,"num_read_kb":19601,"num_write":753,"num_write_kb":10850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.c","version":"143'377","reported_seq":1217,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.744023+0000","last_change":"2024-09-22T01:03:08.754632+0000","last_active":"2024-09-22T01:03:10.744023+0000","last_peered":"2024-09-22T01:03:10.744023+0000","last_clean":"2024-09-22T01:03:10.744023+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:03:10.744023+0000","last_undegraded":"2024-09-22T01:03:10.744023+0000","last_fullsized":"2024-09-22T01:03:10.744023+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:35:32.774495+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0095620649999999998,"stat_sum":{"num_bytes":20594489,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":14,"num_read":2563,"num_read_kb":21660,"num_write":378,"num_write_kb":11071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.d","version":"144'729","reported_seq":5757,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.873739+0000","last_change":"2024-09-22T01:03:08.773213+0000","last_active":"2024-09-22T01:03:09.873739+0000","last_peered":"2024-09-22T01:03:09.873739+0000","last_clean":"2024-09-22T01:03:09.873739+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:03:09.873739+0000","last_undegraded":"2024-09-22T01:03:09.873739+0000","last_fullsized":"2024-09-22T01:03:09.873739+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:57:23.738469+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.018351605,"stat_sum":{"num_bytes":22559112,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":7434,"num_read_kb":29070,"num_write":885,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.e","version":"139'445","reported_seq":4914,"reported_epoch":144,"state":"active+undersized+remapped","last_fresh":"2024-09-22T01:03:09.894807+0000","last_change":"2024-09-22T01:03:09.895417+0000","last_active":"2024-09-22T01:03:09.894807+0000","last_peered":"2024-09-22T01:03:09.894807+0000","last_clean":"2024-09-22T01:03:05.688459+0000","last_became_active":"2024-09-22T01:03:07.787476+0000","last_became_peered":"2024-09-22T01:03:07.787476+0000","last_unstale":"2024-09-22T01:03:09.894807+0000","last_undegraded":"2024-09-22T01:03:09.894807+0000","last_fullsized":"2024-09-22T01:03:07.742415+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.012813236,"stat_sum":{"num_bytes":36911217,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":3823,"num_read_kb":26266,"num_write":500,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","1","4"],"object_location_counts":[{"shards":"7","objects":39}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"4"}]},{"pgid":"3.f","version":"144'359","reported_seq":4854,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.760293+0000","last_change":"2024-09-22T01:03:09.758193+0000","last_active":"2024-09-22T01:03:09.760293+0000","last_peered":"2024-09-22T01:03:09.760293+0000","last_clean":"2024-09-22T01:03:09.760293+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:03:09.760293+0000","last_undegraded":"2024-09-22T01:03:09.760293+0000","last_fullsized":"2024-09-22T01:03:09.760293+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:59:53.988373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025892350000000001,"stat_sum":{"num_bytes":9606686,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":5001,"num_read_kb":10198,"num_write":293,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.10","version":"143'415","reported_seq":3046,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-22T01:03:10.742670+0000","last_change":"2024-09-22T01:03:08.764226+0000","last_active":"2024-09-22T01:03:10.742670+0000","last_peered":"2024-09-22T01:03:10.742670+0000","last_clean":"2024-09-22T01:03:10.742670+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:03:10.742670+0000","last_undegraded":"2024-09-22T01:03:10.742670+0000","last_fullsized":"2024-09-22T01:03:10.742670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:17:19.477833+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022006248999999999,"stat_sum":{"num_bytes":25803863,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":3033,"num_read_kb":14539,"num_write":370,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.11","version":"144'642","reported_seq":2785,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.772539+0000","last_change":"2024-09-22T01:03:09.772539+0000","last_active":"2024-09-22T01:03:09.772539+0000","last_peered":"2024-09-22T01:03:09.772539+0000","last_clean":"2024-09-22T01:03:09.772539+0000","last_became_active":"2024-09-22T01:03:09.761341+0000","last_became_peered":"2024-09-22T01:03:09.761341+0000","last_unstale":"2024-09-22T01:03:09.772539+0000","last_undegraded":"2024-09-22T01:03:09.772539+0000","last_fullsized":"2024-09-22T01:03:09.772539+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:04.306416+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070303320000000002,"stat_sum":{"num_bytes":15555596,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":4508,"num_read_kb":11727,"num_write":781,"num_write_kb":5789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2273258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.12","version":"144'415","reported_seq":3003,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-22T01:03:09.873842+0000","last_change":"2024-09-22T01:03:08.755423+0000","last_active":"2024-09-22T01:03:09.873842+0000","last_peered":"2024-09-22T01:03:09.873842+0000","last_clean":"2024-09-22T01:03:09.873842+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:03:09.873842+0000","last_undegraded":"2024-09-22T01:03:09.873842+0000","last_fullsized":"2024-09-22T01:03:09.873842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:08.444221+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.019560337000000001,"stat_sum":{"num_bytes":14314310,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":21,"num_read":3526,"num_read_kb":7506,"num_write":401,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.13","version":"142'302","reported_seq":6634,"reported_epoch":143,"state":"peering","last_fresh":"2024-09-22T01:03:09.529202+0000","last_change":"2024-09-22T01:03:09.742553+0000","last_active":"2024-09-22T01:03:09.742553+0000","last_peered":"2024-09-22T01:03:09.529202+0000","last_clean":"2024-09-22T01:03:04.977920+0000","last_became_active":"2024-09-22T01:03:07.799577+0000","last_became_peered":"2024-09-22T01:03:07.799577+0000","last_unstale":"2024-09-22T01:03:09.529202+0000","last_undegraded":"2024-09-22T01:03:09.529202+0000","last_fullsized":"2024-09-22T01:03:07.735017+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0089510900000000001,"stat_sum":{"num_bytes":13705818,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":5693,"num_read_kb":26870,"num_write":336,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"4"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":609099528,"num_objects":1016,"num_object_clones":271,"num_object_copies":2032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1016,"num_whiteouts":488,"num_read":196185,"num_read_kb":754420,"num_write":21501,"num_write_kb":410807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4024,"num_bytes_recovered":204797489,"num_keys_recovered":904,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169193472,"data_stored":198584049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":18157,"ondisk_log_size":18157,"up":64,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46556,"kb_used_data":15356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325284,"statfs":{"total":96636764160,"available":96589090816,"internally_reserved":0,"allocated":15724544,"data_stored":16690298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299414,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58560,"kb_used_data":30440,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94313280,"statfs":{"total":96636764160,"available":96576798720,"internally_reserved":0,"allocated":31170560,"data_stored":37325588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77128,"kb_used_data":19568,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94294712,"statfs":{"total":96636764160,"available":96557785088,"internally_reserved":0,"allocated":20037632,"data_stored":23978135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52684,"kb_used_data":25656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319156,"statfs":{"total":96636764160,"available":96582815744,"internally_reserved":0,"allocated":26271744,"data_stored":29294104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90128,"kb_used_data":32572,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94281712,"statfs":{"total":96636764160,"available":96544473088,"internally_reserved":0,"allocated":33353728,"data_stored":35053929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61484,"kb_used_data":12248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310356,"statfs":{"total":96636764160,"available":96573804544,"internally_reserved":0,"allocated":12541952,"data_stored":13976494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64028,"kb_used_data":18952,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94307812,"statfs":{"total":96636764160,"available":96571199488,"internally_reserved":0,"allocated":19406848,"data_stored":23834600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86764,"kb_used_data":25044,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94285076,"statfs":{"total":96636764160,"available":96547917824,"internally_reserved":0,"allocated":25645056,"data_stored":26186228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23330816,"data_stored":24774031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17694720,"data_stored":23018905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10821632,"data_stored":13154665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31633408,"data_stored":34232100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24551424,"data_stored":28472275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29446144,"data_stored":36503740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13991936,"data_stored":15862395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":17723392,"data_stored":22565938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:03:11.907 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [1, 2, 6, 4] 2024-09-22T01:03:11.907 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '1', '2', '6', '4'] 2024-09-22T01:03:11.907 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 1 2 6 4 2024-09-22T01:03:11.971 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:11.972 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:11.972 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:11.972 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:11.985 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:12.029 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:12.029+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:12.104 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:12.100+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:12.150 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:12.150 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:12.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:12.209 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:12.209+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:12.286 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:12.286 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:12.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 160.392578832, 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:12.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:03:12.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008581505, 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:12.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:03:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.00854820000001, 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.300 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008451687, 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:12.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:03:12.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008378436, 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:03:12.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:03:12.314 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008313255, 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:03:12.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008243134, 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:03:12.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008209769, 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008179347, 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:03:12.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008149041, 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:03:12.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008118639, 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.340 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:03:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:03:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:03:12.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.00808784, 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.345 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.00805736700001, 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:03:12.350 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:03:12.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.008026775, 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:03:12.355 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:03:12.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:03:12.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:12.360 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.007988973, 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:12.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:03:12.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:03:12.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:12.365 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.007956608, 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.007926289, 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:03:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.007896397, 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.376 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:03:12.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:03:12.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.380 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.007866638, 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:03:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.00781617299999, 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.386 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:03:12.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:03:12.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.391 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:12.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:12.393 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:12.388+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:12.494 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:12.493+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:12.563 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:12.563 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:12.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:12.621 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:12.621+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:12.695 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:12.693+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:12.742 INFO:teuthology.orchestra.run.smithi002.stderr:set 2.0 pg_upmap_items mapping to [1->2,6->4] 2024-09-22T01:03:12.750 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:12.750 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:12.766 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:03:12.769 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:12.818 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:12.816+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:12.896 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:12.892+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:12.914 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813975482999993, 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:03:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:03:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:03:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813919807999994, 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:03:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813867462999994, 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:03:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:03:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813809551999995, 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:03:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813750913000007, 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:03:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813695680999999, 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:03:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813641957000002, 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:03:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:03:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813570221999996, 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813512841000005, 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:03:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:03:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813462575000003, 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:03:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:03:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813383517999995, 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:03:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:03:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813259269, 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:03:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:03:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:03:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813178687000004, 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.813113498999996, 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.812338530000005, 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.812288745000004, 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:03:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:03:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.812207643999997, 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:03:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:03:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.812137659000001, 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:03:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.812060294999995, 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:03:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:03:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:03:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 92.811999774, 2024-09-22T01:03:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:03:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:03:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:03:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:13.065 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:13.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:13.066 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:13.067 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:13.065+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:13.067 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:13.082 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:13.082 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:13.102 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:13.102 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:13.120 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:13.120 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:13.141 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:13.141 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:13.163 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:13.167 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:13.165+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:13.185 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:13.185 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:13.191 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:13.191 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:13.206 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:13.206 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:13.207 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:13.268 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:13.264+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:13.326 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:13.326 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:13.340 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:13.382 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:13.380+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:13.470 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:13.469+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:13.507 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:13.507 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:13.521 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:13.571 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:13.569+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:13.648 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:13.648 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:13.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:13.672 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:13.669+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:13.738 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:03:13.773 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:13.773+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:13.807 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:13.874 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:13.873+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:13.974 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:13.972+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:13.985 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:13.985 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:14.000 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:14.076 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:14.073+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:14.171 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:14.172 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:14.176 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:14.173+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:14.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:14.278 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:14.276+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:14.310 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:14.311 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:14.325 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:14.410 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:14.408+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:14.479 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:14.476+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:14.491 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:14.492 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:14.506 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:14.594 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:14.592+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:14.671 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:14.671 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:14.680 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:14.676+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:14.685 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:14.782 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:14.780+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:14.863 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:14.863 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:14.877 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:14.922 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:14.920+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:14.984 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:14.980+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:15.067 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:15.067 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:15.084 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:15.126 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:15.124+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:15.186 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:15.185+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:15.223 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:15.223 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:15.226 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:15.238 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:15.287 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:15.284+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:15.386 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:15.387 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:15.388 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:15.385+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:15.403 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:15.403 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:15.405 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:15.417 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:15.424 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:15.424 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:15.444 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:15.444 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:15.463 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:15.464 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:15.483 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:15.483 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:15.502 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:15.503 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:15.505 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:15.505+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:15.521 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:15.521 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:15.580 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:15.580 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:15.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:15.580 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:15.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.580 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31990 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.580 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:15.580 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.609403118, 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081263087999999997, 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31990 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968990+0000", 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968991+0000", 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.063e-06 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.968997+0000", 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5160000000000002e-06 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969014+0000", 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7068999999999999e-05 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969027+0000", 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3166000000000001e-05 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050241+0000", 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081213863999999997 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050253+0000", 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.241e-05 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31991 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.584 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.609350595, 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081294027000000005, 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31991 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969043+0000", 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969044+0000", 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-06 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969049+0000", 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2170000000000002e-06 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969066+0000", 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969079+0000", 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2876e-05 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050319+0000", 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081240827000000002 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050337+0000", 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7235e-05 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31992 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.588 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.609298274, 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081307669999999999, 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31992 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969095+0000", 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969096+0000", 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.026e-06 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969112+0000", 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5773e-05 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969129+0000", 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6922000000000002e-05 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969142+0000", 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3142000000000001e-05 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050389+0000", 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081246992000000004 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050403+0000", 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3815000000000001e-05 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31994 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.592 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.6091325, 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081259283000000002, 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31994 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969261+0000", 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969262+0000", 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3300000000000001e-07 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969265+0000", 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969274+0000", 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3489999999999998e-06 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969281+0000", 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7830000000000001e-06 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050500+0000", 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081219360000000004 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050520+0000", 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9961000000000001e-05 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:31995 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.596 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.60910407199999, 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081308264000000005, 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 31995 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969289+0000", 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969290+0000", 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3300000000000001e-07 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969293+0000", 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000001e-06 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969302+0000", 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7960000000000001e-06 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969309+0000", 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050579+0000", 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081270437000000001 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050597+0000", 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8385000000000001e-05 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.600 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32013 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.60907626300001, 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081345807000000006, 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32013 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969317+0000", 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969318+0000", 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969321+0000", 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969330+0000", 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7029999999999995e-06 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969337+0000", 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.883e-06 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050652+0000", 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081315163999999995 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050663+0000", 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0968000000000001e-05 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32014 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:15.604 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.60904815799999, 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397088000000006, 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32014 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969345+0000", 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969346+0000", 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969349+0000", 2024-09-22T01:03:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1530000000000001e-06 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969358+0000", 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7930000000000003e-06 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969364+0000", 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6989999999999997e-06 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050723+0000", 2024-09-22T01:03:15.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081358877999999996 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050742+0000", 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8955000000000001e-05 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32015 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.609020616, 2024-09-22T01:03:15.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081417011999999997, 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32015 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969373+0000", 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969376+0000", 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9529999999999999e-06 2024-09-22T01:03:15.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969385+0000", 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969392+0000", 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.956e-06 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050781+0000", 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081389258000000006 2024-09-22T01:03:15.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050790+0000", 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5560000000000008e-06 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32016 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.608992968, 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422597999999999, 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32016 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969400+0000", 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969401+0000", 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969404+0000", 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9000000000000002e-06 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969413+0000", 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6859999999999995e-06 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969419+0000", 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8199999999999999e-06 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050816+0000", 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081396483000000006 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050823+0000", 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0960000000000003e-06 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32017 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.60896582300001, 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423671000000003, 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.616 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32017 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969427+0000", 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969428+0000", 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969431+0000", 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.92e-06 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969440+0000", 2024-09-22T01:03:15.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8230000000000001e-06 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969447+0000", 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1459999999999999e-06 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050844+0000", 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397284 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050851+0000", 2024-09-22T01:03:15.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32018 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.608937864, 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081423593000000002, 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32018 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969455+0000", 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969456+0000", 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969459+0000", 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2030000000000001e-06 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969468+0000", 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8589999999999994e-06 2024-09-22T01:03:15.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969475+0000", 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6969999999999996e-06 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050872+0000", 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397272000000007 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050879+0000", 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.922e-06 2024-09-22T01:03:15.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32019 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.608910049, 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081422869999999994, 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32019 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969483+0000", 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969484+0000", 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2900000000000003e-07 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969487+0000", 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.757e-06 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969495+0000", 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4400000000000005e-06 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969502+0000", 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050900+0000", 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081397554999999996 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050906+0000", 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5629999999999997e-06 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32020 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.608883104, 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427278000000006, 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32020 2024-09-22T01:03:15.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969510+0000", 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969511+0000", 2024-09-22T01:03:15.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969514+0000", 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969522+0000", 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999992e-06 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969529+0000", 2024-09-22T01:03:15.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050929+0000", 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081399641999999994 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050937+0000", 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32021 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.60885540300001, 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081427445000000001, 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32021 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969538+0000", 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969539+0000", 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969550+0000", 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1647e-05 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969559+0000", 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2890000000000003e-06 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969635+0000", 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5536000000000003e-05 2024-09-22T01:03:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050958+0000", 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081323254999999997 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050965+0000", 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.635 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32022 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.608740979, 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081340155999999997, 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32022 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969652+0000", 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969653+0000", 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969658+0000", 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7029999999999997e-06 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969675+0000", 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969683+0000", 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050986+0000", 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081303159 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.050992+0000", 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32023 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.60870162099999, 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081328867999999999, 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32023 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:15.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969692+0000", 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969696+0000", 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1930000000000002e-06 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969706+0000", 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0546e-05 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969713+0000", 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051014+0000", 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081300725000000004 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051021+0000", 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6610000000000003e-06 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32024 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.643 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.608671466, 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081325718000000005, 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32024 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:15.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969722+0000", 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2300000000000001e-07 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969725+0000", 2024-09-22T01:03:15.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8870000000000001e-06 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969736+0000", 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0522e-05 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969743+0000", 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051041+0000", 2024-09-22T01:03:15.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081298322000000006 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051048+0000", 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3500000000000002e-06 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32025 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.60864220800001, 2024-09-22T01:03:15.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081351182999999994, 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32025 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969751+0000", 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969752+0000", 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969755+0000", 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-06 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969764+0000", 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000007e-06 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969771+0000", 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8519999999999998e-06 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051085+0000", 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081314400999999994 2024-09-22T01:03:15.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051102+0000", 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7263000000000001e-05 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32026 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.608614332, 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081392545999999996, 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32026 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969779+0000", 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969780+0000", 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969782+0000", 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8959999999999999e-06 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969791+0000", 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969800+0000", 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7269999999999996e-06 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051155+0000", 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081355372999999995 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051172+0000", 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6280999999999999e-05 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:32027 3.1c 3.57f6bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.60858495799999, 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081444120999999994, 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:15.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 32027 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969808+0000", 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:15.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969809+0000", 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3e-07 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969812+0000", 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969821+0000", 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:15.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:16.969828+0000", 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051235+0000", 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.081406966999999997 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:17.051252+0000", 2024-09-22T01:03:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7921e-05 2024-09-22T01:03:15.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:15.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:15.659 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:15.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:15.662 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:15.660+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:15.761 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:15.756+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:15.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:15.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:15.893 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:15.893+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:15.963 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:15.961+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:15.966 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:15.967 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:15.981 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:16.069 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:16.069+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:16.164 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.805037591000001, 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.165 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:03:16.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.169 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.805000621999994, 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:16.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804972352999997, 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804934477000003, 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:03:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804895887000001, 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:03:16.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804823558999999, 2024-09-22T01:03:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:03:16.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804785692999999, 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:03:16.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:16.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804743454000004, 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:16.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:16.200 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804689069000005, 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804588304000006, 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:03:16.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804501379000001, 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.209 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:03:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.804443054999993, 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.217 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.803989904999995, 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:03:16.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.803351418999995, 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.222 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.226 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.803063821999999, 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:03:16.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.802460852999999, 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:03:16.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:03:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:03:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:03:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:03:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.801380554000005, 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.801310676, 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:03:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.800748685000002, 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:03:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:151553 3.a 3:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e146)", 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:03:11.763113+0000", 2024-09-22T01:03:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.3984069740000002, 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.99049487899999999, 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 151553 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763113+0000", 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763113+0000", 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763115+0000", 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0310000000000001e-06 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763123+0000", 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763139+0000", 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6507999999999998e-05 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763151+0000", 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1308000000000001e-05 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763235+0000", 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4133000000000002e-05 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763250+0000", 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5376999999999999e-05 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:12.753299+0000", 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.99004856399999996 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:12.753457+0000", 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015883 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:12.753608+0000", 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000150688 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.255 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:16.255 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:16.161+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:16.256 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:16.266 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:16.264+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:16.367 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:16.364+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:16.430 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:16.430 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:16.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:16.489 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:16.489+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:16.567 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:16.567 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:16.567 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:16.567 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:16.567 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:16.568 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:16.564+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:16.581 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:16.669 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:16.664+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:16.745 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 164.851568665, 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:03:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:03:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:03:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.467571338, 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:03:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:03:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.756 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.467538033, 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:16.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:03:16.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:03:16.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.46744151999999, 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:03:16.760 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:03:16.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.467368269, 2024-09-22T01:03:16.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:16.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:03:16.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.768 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.46730308799999, 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:03:16.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:03:16.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.467232967, 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:03:16.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:16.777 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.46719960199999, 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:16.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:03:16.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:16.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.46716918, 2024-09-22T01:03:16.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.467138874, 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.789 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.46710847200001, 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:03:16.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.46707767300001, 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:03:16.794 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:03:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:03:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.4670472, 2024-09-22T01:03:16.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:03:16.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:03:16.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.46701660799999, 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:03:16.802 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:03:16.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.466978806, 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:03:16.806 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:03:16.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.466946441, 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:03:16.810 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.466916122, 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:03:16.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:03:16.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.46688623, 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.466856471, 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:03:16.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.466806006, 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:16.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.829 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:16.830 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:16.830 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:16.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:16.832 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:16.828+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:16.932 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:16.928+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:16.958 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:16.959 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:16.959 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:16.959 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:16.974 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:17.034 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:17.032+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:17.040 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:03:17.098 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:17.098 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:17.098 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:17.098 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:17.098 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:17.114 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:17.134 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:17.132+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:17.235 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:17.232+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107714830000006, 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.240 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:03:17.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107659154999993, 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:03:17.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107606809999993, 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:03:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107548898999994, 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:03:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107490260000006, 2024-09-22T01:03:17.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.266 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107435027999998, 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:03:17.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107381304, 2024-09-22T01:03:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107309568999995, 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:03:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107252188000004, 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:03:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107201922000002, 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:03:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:03:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.107122864999994, 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:03:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.106998615999998, 2024-09-22T01:03:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:03:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.106918034000003, 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:03:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.106852845999995, 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:03:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.106077877000004, 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:03:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.106028092000003, 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:03:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.105946990999996, 2024-09-22T01:03:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:03:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.105877006, 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:03:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.105799641999994, 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:03:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:03:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:03:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 97.105739120999999, 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:03:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:03:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:03:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:17.346 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:17.348 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:17.345+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:17.348 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:17.449 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:17.449+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:17.484 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:17.484 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:17.484 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:17.484 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:17.498 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:17.541 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:17.550 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:17.549+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:17.623 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:17.624 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:17.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:17.682 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:17.680+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:17.696 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:17.696 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:17.714 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:17.731 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:17.731 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:17.748 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:17.748 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:17.751 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:17.748+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:17.770 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:17.770 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:17.771 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:03:17.771 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:03:17.771 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-22T01:03:17.771 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:03:17.772 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:17.772 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:17.787 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:17.792 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:17.792 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:17.813 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:17.813 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:17.850 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:17.851 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:17.849+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:17.910 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:17.910 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:17.926 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:17.952 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:17.949+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:18.055 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:18.048+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:18.099 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:18.099 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:03:18.104 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 152, 2024-09-22T01:03:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:03:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-22T01:00:18.015746+0000", 2024-09-22T01:03:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-22T01:03:18.040027+0000", 2024-09-22T01:03:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-22T01:01:26.960327+0000", 2024-09-22T01:03:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-22T01:03:05.961739+0000", 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5804032, 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "nodeep-scrub", 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:03:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-22T01:03:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:30.699559+0000", 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:36.410605+0000", 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 2, 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 21, 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9900000095367432, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9900000095367432, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:03:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:46.342370+0000", 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "152", 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 50, 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 152, 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:03:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.75, 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.75, 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "8c0f94a8-3c63-442d-9dc1-50c1278929da", 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 146, 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2277223948", 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2277223948", 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2277223948", 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2277223948", 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "2d2cf30e-e9bd-40ba-8293-193faa02ea69", 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 142, 2024-09-22T01:03:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/463340194", 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/463340194", 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/463340194", 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/463340194", 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a3d3e330-9ea1-4a5c-811b-f285f9a3cff4", 2024-09-22T01:03:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 143, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-22T01:03:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-22T01:03:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-22T01:03:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-22T01:03:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/387643397", 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/387643397", 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/387643397", 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/387643397", 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "239072f0-4b69-4624-ba6c-733f299208da", 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 144, 2024-09-22T01:03:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1215210764", 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1215210764", 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1215210764", 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1215210764", 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fd35cb24-eb2c-4c17-abe9-ce5c34db9014", 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 146, 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6800", 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6801", 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6802", 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6803", 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6806", 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6807", 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6804", 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6805", 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6801/2463696152", 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6803/2463696152", 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6807/2463696152", 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6805/2463696152", 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "b60a584d-5334-41e4-92d9-f6689de6b8bf", 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 15, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 25, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 52, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 145, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 26, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6824", 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6825", 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6826", 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6827", 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6830", 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6831", 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6828", 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6829", 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6825/1737475131", 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6827/1737475131", 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1737475131", 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1737475131", 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:03:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "9be24894-1100-4e4e-bf1b-79e163fd97e0", 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 146, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6816", 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6817", 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6818", 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6819", 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6822", 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6823", 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6820", 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6821", 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6817/4220217922", 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6819/4220217922", 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6823/4220217922", 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6821/4220217922", 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "293cfbc3-8c2e-40b5-ab89-0a124cc8651c", 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 147, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6808", 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6809", 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6810", 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6811", 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6814", 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6815", 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6812", 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6813", 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6809/3867156106", 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6811/3867156106", 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3867156106", 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3867156106", 2024-09-22T01:03:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.556128+0000", 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.159753+0000", 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:03:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.766199+0000", 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.155529+0000", 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.226232+0000", 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.243346+0000", 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 26 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.143084+0000", 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.524369+0000", 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.11", 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: 4, 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: 6 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.17", 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: 1 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [ 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "2.0", 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 1, 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 2 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.13", 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 7, 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 3 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-22T01:03:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [ 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 24, 2024-09-22T01:03:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 7 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [ 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 24, 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 1 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:18.181 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:18.183 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:18.184 INFO:tasks.thrashosds.thrasher:j is {'epoch': 152, 'fsid': '7d08b428-4185-4f0f-a0fc-f7dc1fefdce9', 'created': '2024-09-22T01:00:18.015746+0000', 'modified': '2024-09-22T01:03:18.040027+0000', 'last_up_change': '2024-09-22T01:01:26.960327+0000', 'last_in_change': '2024-09-22T01:03:05.961739+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:00:30.699559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:00:36.410605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:00:46.342370+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 152, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8c0f94a8-3c63-442d-9dc1-50c1278929da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2277223948}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2277223948}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2277223948}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2277223948}]}, 'public_addr': '172.21.15.2:6825/2277223948', 'cluster_addr': '172.21.15.2:6827/2277223948', 'heartbeat_back_addr': '172.21.15.2:6831/2277223948', 'heartbeat_front_addr': '172.21.15.2:6829/2277223948', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 463340194}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 463340194}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 463340194}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 463340194}]}, 'public_addr': '172.21.15.2:6817/463340194', 'cluster_addr': '172.21.15.2:6819/463340194', 'heartbeat_back_addr': '172.21.15.2:6823/463340194', 'heartbeat_front_addr': '172.21.15.2:6821/463340194', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 387643397}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 387643397}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 387643397}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 387643397}]}, 'public_addr': '172.21.15.2:6801/387643397', 'cluster_addr': '172.21.15.2:6803/387643397', 'heartbeat_back_addr': '172.21.15.2:6807/387643397', 'heartbeat_front_addr': '172.21.15.2:6805/387643397', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '239072f0-4b69-4624-ba6c-733f299208da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1215210764}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1215210764}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1215210764}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1215210764}]}, 'public_addr': '172.21.15.2:6809/1215210764', 'cluster_addr': '172.21.15.2:6811/1215210764', 'heartbeat_back_addr': '172.21.15.2:6815/1215210764', 'heartbeat_front_addr': '172.21.15.2:6813/1215210764', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 2463696152}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 2463696152}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 2463696152}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 2463696152}]}, 'public_addr': '172.21.15.119:6801/2463696152', 'cluster_addr': '172.21.15.119:6803/2463696152', 'heartbeat_back_addr': '172.21.15.119:6807/2463696152', 'heartbeat_front_addr': '172.21.15.119:6805/2463696152', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b60a584d-5334-41e4-92d9-f6689de6b8bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 25, 'up_from': 52, 'up_thru': 145, 'down_at': 26, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1737475131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1737475131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1737475131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1737475131}]}, 'public_addr': '172.21.15.119:6825/1737475131', 'cluster_addr': '172.21.15.119:6827/1737475131', 'heartbeat_back_addr': '172.21.15.119:6831/1737475131', 'heartbeat_front_addr': '172.21.15.119:6829/1737475131', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9be24894-1100-4e4e-bf1b-79e163fd97e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 4220217922}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 4220217922}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 4220217922}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 4220217922}]}, 'public_addr': '172.21.15.119:6817/4220217922', 'cluster_addr': '172.21.15.119:6819/4220217922', 'heartbeat_back_addr': '172.21.15.119:6823/4220217922', 'heartbeat_front_addr': '172.21.15.119:6821/4220217922', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '293cfbc3-8c2e-40b5-ab89-0a124cc8651c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 147, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3867156106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3867156106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3867156106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3867156106}]}, 'public_addr': '172.21.15.119:6809/3867156106', 'cluster_addr': '172.21.15.119:6811/3867156106', 'heartbeat_back_addr': '172.21.15.119:6815/3867156106', 'heartbeat_front_addr': '172.21.15.119:6813/3867156106', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.556128+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.159753+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.766199+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.155529+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.226232+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.243346+0000', 'dead_epoch': 26}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.143084+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.524369+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.11', 'osds': [4, 6]}, {'pgid': '3.17', 'osds': [6, 1]}], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '3.13', 'mappings': [{'from': 7, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 7}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:03:18.185 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.11 2024-09-22T01:03:18.185 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.11 2024-09-22T01:03:18.185 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:18.180+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:18.283 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:18.281+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:18.364 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:18.365 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:18.381 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:18.384 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:18.381+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:18.484 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:18.480+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:18.498 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:18.498 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:18.512 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:18.598 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:18.596+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:18.674 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:18.674 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:18.686 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:18.684+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:18.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:18.787 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:18.785+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:18.856 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:18.856 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:18.870 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:18.914 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:18.912+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:18.988 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:18.989+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:18.990 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:18.990 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:19.006 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:clear 3.11 pg_upmap mapping 2024-09-22T01:03:19.076 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:03:19.103 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:19.100+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:19.190 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:19.188+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:19.195 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:19.195 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:19.212 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:19.302 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:19.300+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:19.374 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:19.374 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:19.387 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:19.433 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:19.433+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:19.492 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:19.489+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:19.519 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:19.520 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:19.534 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:19.621 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:19.621+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:19.693 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:19.693+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:19.694 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:19.694 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:19.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:19.795 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:19.792+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:19.873 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:19.873 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:19.876 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:19.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:19.922 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:19.920+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:19.997 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:19.997+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:20.006 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:20.007 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:20.021 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:20.035 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:20.035 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:20.051 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:20.052 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:20.070 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:20.070 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:03:20.095 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:20.096 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:20.098 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:20.096+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:20.113 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:20.114 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:20.133 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:20.134 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:20.152 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:20.153 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:20.171 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:20.171 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:20.202 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:20.202 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:20.202 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:20.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:20.230 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:20.228+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:20.300 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:20.296+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:20.341 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:20.341 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74349 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6639616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982608146000004, 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097443432999999996, 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74349 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356483+0000", 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356486+0000", 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9239999999999999e-06 2024-09-22T01:03:20.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356495+0000", 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3229999999999998e-06 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356510+0000", 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5636000000000001e-05 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453733+0000", 2024-09-22T01:03:20.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097222997000000005 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453799+0000", 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6094999999999997e-05 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453926+0000", 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012683799999999999 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74351 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982571176999997, 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097681670999999998, 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74351 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356520+0000", 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:03:20.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356523+0000", 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356532+0000", 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1069999999999994e-06 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356539+0000", 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7530000000000004e-06 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.453987+0000", 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097447581000000005 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454094+0000", 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107471 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454201+0000", 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107142 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74353 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6696960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982542907999999, 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097828347999999996, 2024-09-22T01:03:20.351 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74353 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356548+0000", 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0399999999999996e-07 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356560+0000", 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1579999999999999e-05 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356569+0000", 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2629999999999992e-06 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356576+0000", 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.737e-06 2024-09-22T01:03:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454253+0000", 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097676896999999999 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454313+0000", 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0270999999999999e-05 2024-09-22T01:03:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454376+0000", 2024-09-22T01:03:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2996000000000005e-05 2024-09-22T01:03:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74355 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6725632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982505032000006, 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098070117999999998, 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74355 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356586+0000", 2024-09-22T01:03:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356589+0000", 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8930000000000001e-06 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356598+0000", 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6200000000000005e-06 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356615+0000", 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6969999999999998e-05 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454434+0000", 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097819516999999995 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454569+0000", 2024-09-22T01:03:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013433100000000001 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454656+0000", 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7159999999999999e-05 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74357 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6754304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982466442000003, 2024-09-22T01:03:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098213086000000005, 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74357 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356624+0000", 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356625+0000", 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8299999999999996e-07 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356628+0000", 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-09-22T01:03:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356637+0000", 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0170000000000002e-06 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356644+0000", 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8260000000000004e-06 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454717+0000", 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098073467999999997 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454777+0000", 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9806999999999998e-05 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454837+0000", 2024-09-22T01:03:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0238999999999997e-05 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74359 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982394114000002, 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098399373999999998, 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74359 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356697+0000", 2024-09-22T01:03:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356698+0000", 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.15e-06 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356702+0000", 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356713+0000", 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519999999999999e-05 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356720+0000", 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7560000000000002e-06 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454894+0000", 2024-09-22T01:03:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098174213999999996 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.454987+0000", 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3067999999999997e-05 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455096+0000", 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010926599999999999 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74361 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6811648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982356248000002, 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098593022000000002, 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74361 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356735+0000", 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9999999999999996e-07 2024-09-22T01:03:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356740+0000", 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356757+0000", 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6806000000000001e-05 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356766+0000", 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8999999999999995e-06 2024-09-22T01:03:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455163+0000", 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397826999999993 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455265+0000", 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101236 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455328+0000", 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999994e-05 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74362 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6840320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982314009000007, 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098812997999999999, 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74362 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356777+0000", 2024-09-22T01:03:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356778+0000", 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7300000000000005e-07 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356783+0000", 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0429999999999998e-06 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356800+0000", 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7252999999999999e-05 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356813+0000", 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455387+0000", 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098574017 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455508+0000", 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012126299999999999 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455590+0000", 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1539000000000007e-05 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74364 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6868992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982259623999994, 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099035605999999998, 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74364 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356831+0000", 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356832+0000", 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.207e-06 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356853+0000", 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0593000000000001e-05 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356892+0000", 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8625999999999999e-05 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356910+0000", 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8362000000000001e-05 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455640+0000", 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098729872999999996 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455754+0000", 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113966 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455867+0000", 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112979 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74366 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982158858999995, 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099207737000000004, 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74366 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356932+0000", 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356933+0000", 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.173e-06 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356940+0000", 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356962+0000", 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1719999999999999e-05 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.356998+0000", 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6689000000000001e-05 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.455924+0000", 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098925974 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456022+0000", 2024-09-22T01:03:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7788000000000006e-05 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456140+0000", 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011769699999999999 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74368 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6926336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982071934000004, 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099289766000000002, 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74368 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357019+0000", 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357020+0000", 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357026+0000", 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9569999999999999e-06 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357044+0000", 2024-09-22T01:03:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8439000000000001e-05 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357058+0000", 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3913999999999999e-05 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456187+0000", 2024-09-22T01:03:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099129019999999998 2024-09-22T01:03:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456245+0000", 2024-09-22T01:03:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7323e-05 2024-09-22T01:03:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456309+0000", 2024-09-22T01:03:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4023999999999996e-05 2024-09-22T01:03:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74369 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.982013609999996, 2024-09-22T01:03:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099501316000000006, 2024-09-22T01:03:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74369 2024-09-22T01:03:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357077+0000", 2024-09-22T01:03:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357469+0000", 2024-09-22T01:03:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00039134199999999998 2024-09-22T01:03:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357476+0000", 2024-09-22T01:03:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6090000000000003e-06 2024-09-22T01:03:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357498+0000", 2024-09-22T01:03:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2184e-05 2024-09-22T01:03:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357510+0000", 2024-09-22T01:03:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-09-22T01:03:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456368+0000", 2024-09-22T01:03:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098858420000000002 2024-09-22T01:03:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456473+0000", 2024-09-22T01:03:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010498800000000001 2024-09-22T01:03:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456579+0000", 2024-09-22T01:03:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105227 2024-09-22T01:03:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74370 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 6983680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.981560459999997, 2024-09-22T01:03:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.099217300999999994, 2024-09-22T01:03:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74370 2024-09-22T01:03:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.357530+0000", 2024-09-22T01:03:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358008+0000", 2024-09-22T01:03:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047754600000000003 2024-09-22T01:03:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358018+0000", 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358081+0000", 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2432000000000001e-05 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358100+0000", 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.969e-05 2024-09-22T01:03:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456629+0000", 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098528800999999999 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456688+0000", 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9070000000000002e-05 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456748+0000", 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9438999999999998e-05 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74371 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.980921973999997, 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098857306000000006, 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74371 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358169+0000", 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358354+0000", 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018502899999999999 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358364+0000", 2024-09-22T01:03:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0046e-05 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358389+0000", 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4873000000000001e-05 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358406+0000", 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7343e-05 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456803+0000", 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098397334000000003 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.456894+0000", 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0124000000000007e-05 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457026+0000", 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000132557 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74372 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7041024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.980634377000001, 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098741176, 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74372 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358456+0000", 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358953+0000", 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049649899999999996 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.358977+0000", 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3694999999999999e-05 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359006+0000", 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359025+0000", 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8275999999999999e-05 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457076+0000", 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098051632 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457136+0000", 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9312999999999999e-05 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457198+0000", 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1910999999999997e-05 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74374 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7069696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.980031408000002, 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.098409849999999993, 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74374 2024-09-22T01:03:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.359059+0000", 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360036+0000", 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00097661100000000006 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360047+0000", 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1314e-05 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360084+0000", 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6949000000000001e-05 2024-09-22T01:03:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360107+0000", 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457254+0000", 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097147009000000006 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457370+0000", 2024-09-22T01:03:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011576100000000001 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457469+0000", 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9253999999999995e-05 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74375 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7098368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.978951108999993, 2024-09-22T01:03:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097621488000000006, 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74375 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360140+0000", 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360141+0000", 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.437e-06 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360148+0000", 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1060000000000002e-06 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360169+0000", 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.031e-05 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360184+0000", 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5342999999999999e-05 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457535+0000", 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097351278999999999 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457673+0000", 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137918 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457761+0000", 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8095000000000003e-05 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74376 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.978881231000003, 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097813391999999999, 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74376 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360210+0000", 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360674+0000", 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00046407300000000001 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360682+0000", 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.96e-06 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360702+0000", 2024-09-22T01:03:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0013e-05 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360713+0000", 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.188e-05 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457844+0000", 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097130741000000007 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.457941+0000", 2024-09-22T01:03:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7140999999999997e-05 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458023+0000", 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1583999999999997e-05 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:74378 3.a 3:57e38b21:::rbd_data.117cf172dcd2.0000000000000002:1e [read 7155712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.978319240000005, 2024-09-22T01:03:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.097497555999999999, 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 74378 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.360772+0000", 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361012+0000", 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024079099999999999 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361022+0000", 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5570000000000001e-06 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361047+0000", 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4658999999999999e-05 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.361115+0000", 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8365e-05 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458044+0000", 2024-09-22T01:03:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.096928748999999995 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458136+0000", 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2312000000000007e-05 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:54.458269+0000", 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013312299999999999 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:151553 3.a 3:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e146)", 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:03:11.763113+0000", 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5759775289999993, 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.99049487899999999, 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 151553 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763113+0000", 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763113+0000", 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763115+0000", 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0310000000000001e-06 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763123+0000", 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763139+0000", 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6507999999999998e-05 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763151+0000", 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1308000000000001e-05 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763235+0000", 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4133000000000002e-05 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:11.763250+0000", 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5376999999999999e-05 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:12.753299+0000", 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.99004856399999996 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:12.753457+0000", 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015883 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:03:12.753608+0000", 2024-09-22T01:03:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000150688 2024-09-22T01:03:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.449 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:20.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:20.451 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:20.448+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:20.552 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:20.548+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:20.620 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:20.620 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:20.634 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:20.652 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:20.648+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:20.753 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:20.748+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:20.758 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:20.759 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:20.759 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:20.759 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:20.759 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:20.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:20.854 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:20.852+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:20.896 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 169.00274470100001, 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:03:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:03:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:03:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:03:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.61874737399999, 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.61871406900001, 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:03:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:03:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618617556, 2024-09-22T01:03:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:03:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:03:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618544305, 2024-09-22T01:03:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:03:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618479124, 2024-09-22T01:03:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:03:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618409003, 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618375638, 2024-09-22T01:03:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:03:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.61834521599999, 2024-09-22T01:03:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:03:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.61831491, 2024-09-22T01:03:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:03:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:03:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618284508, 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:03:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:03:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618253709, 2024-09-22T01:03:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:03:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.61822323600001, 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:03:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:03:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618192644, 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:03:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:03:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618154842, 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:03:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:03:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618122477, 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:03:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618092158, 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618062266, 2024-09-22T01:03:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.618032507, 2024-09-22T01:03:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:03:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.61798204199999, 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:03:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:20.994 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:20.996 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:20.997 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:20.992+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:21.097 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:21.096+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:21.180 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:21.180 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:21.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:21.198 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:21.196+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:21.298 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:21.296+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:21.374 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:21.375 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:21.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:21.433 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:21.432+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:21.500 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:21.496+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:21.557 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42547620000001, 2024-09-22T01:03:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:03:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42542052500001, 2024-09-22T01:03:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:03:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:03:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42536818000001, 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:03:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42531026899999, 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:03:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:03:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:03:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42525163000001, 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:03:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.425196398, 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:03:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:03:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.425142674, 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:03:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:03:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:03:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42507093899999, 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:03:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:03:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.425013558, 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.424963292, 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:03:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:03:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:03:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42488423499999, 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:03:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.424759986, 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:03:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:03:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:03:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.424679404, 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:03:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:03:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:03:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42461421599999, 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.423839247, 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:03:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:03:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.423789462, 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:03:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:03:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:03:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.423708361, 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:03:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:03:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.423638376, 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:03:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:03:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.42356101199999, 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 101.423500491, 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:03:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:21.675 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:21.677 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:21.677 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:21.672+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:21.778 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:21.776+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:21.811 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:21.811 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:21.811 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:21.812 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:21.827 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:21.918 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:21.916+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:21.980 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:21.976+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:21.997 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:21.997 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:21.997 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:21.997 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:21.997 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:22.011 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:22.098 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:03:22.102 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:22.100+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:22.181 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:22.176+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:22.188 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:22.199 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 100.699122321, 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 100.69909447800001, 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:03:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 100.699062262, 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:03:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:03:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 100.699040524, 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:03:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:03:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 100.69902086899999, 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 100.69898985499999, 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:03:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:03:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:03:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743585644000007, 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:03:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:03:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:03:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743546008999999, 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:03:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:03:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743458567999994, 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:03:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743418348999995, 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:03:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:03:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743382456999996, 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:03:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:03:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:03:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:03:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:03:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743334809000004, 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:03:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:03:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743300039999994, 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743266847000001, 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:03:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743222633000002, 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743180444000004, 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:03:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:03:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743146331999995, 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:03:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:03:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:03:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:03:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743113405000003, 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:03:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.743078636999996, 2024-09-22T01:03:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:151264 3.1b 3:deb34d03:::rbd_object_map.117cf172dcd2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.204889884, 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.812141679, 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 151264 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993280+0000", 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8669999999999998e-06 2024-09-22T01:03:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993295+0000", 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5235999999999999e-05 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993321+0000", 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5775e-05 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993335+0000", 2024-09-22T01:03:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4306e-05 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993415+0000", 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9975999999999995e-05 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993426+0000", 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1063000000000001e-05 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978616+0000", 2024-09-22T01:03:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98518988299999999 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978652+0000", 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5976000000000002e-05 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.803938+0000", 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.82528633799999995 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804052+0000", 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113491 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804224+0000", 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172375 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805210+0000", 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000986385 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805298+0000", 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7410000000000005e-05 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805343+0000", 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5451999999999998e-05 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805418+0000", 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5146000000000002e-05 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:22.307 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:22.308 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:22.308 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:22.304+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:22.343 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:22.344 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:22.360 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:22.361 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:22.377 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:22.378 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:22.394 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:22.394 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:22.409 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:22.408+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:22.415 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:22.415 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:22.436 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:22.436 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:22.456 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:22.457 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:22.460 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:22.461 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:22.475 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:22.477 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:22.478 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:22.510 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:22.508+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:22.594 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:22.594 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:22.610 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:22.611 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:22.608+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:22.712 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:22.708+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:03:22.745 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:22.745 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:22.767 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:22.813 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:22.812+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:22.914 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:22.912+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:22.943 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:22.944 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:22.959 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:23.015 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:23.012+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:23.116 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:23.112+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:23.130 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:23.130 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:23.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:23.230 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:23.228+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:23.312 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:23.312 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:23.318 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:23.316+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:23.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:23.419 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:23.416+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:23.499 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:23.499 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:23.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:23.558 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:23.556+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:23.622 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:23.620+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:23.637 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:23.637 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:23.652 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:23.723 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:23.720+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:23.826 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:23.824+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:23.827 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:23.828 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:23.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:23.927 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:23.924+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:24.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:24.045 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:24.044+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:24.104 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:03:24.104 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:03:24.104 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-22T01:03:24.104 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:03:24.150 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:24.148+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:24.169 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:24.169 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:24.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:24.230 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:24.228+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:24.313 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:24.313 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:24.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:24.369 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:24.368+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 158, 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "7d08b428-4185-4f0f-a0fc-f7dc1fefdce9", 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-22T01:00:18.015746+0000", 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-22T01:03:23.735725+0000", 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-22T01:01:26.960327+0000", 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-22T01:03:05.961739+0000", 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5801984, 2024-09-22T01:03:24.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "noscrub", 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-22T01:03:24.424 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:30.699559+0000", 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:24.426 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-22T01:03:24.427 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:24.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:24.429 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-22T01:03:24.430 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:36.410605+0000", 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:03:24.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-22T01:03:24.432 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:24.433 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 2, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 21, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:24.434 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:24.435 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9900000095367432, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9900000095367432, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:03:24.436 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-22T01:00:46.342370+0000", 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-22T01:03:24.437 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-22T01:03:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-22T01:03:24.439 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-22T01:03:24.439 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-22T01:03:24.439 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-22T01:03:24.439 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-22T01:03:24.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.439 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "158", 2024-09-22T01:03:24.439 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-22T01:03:24.439 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:03:24.441 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:03:24.441 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-22T01:03:24.441 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 50, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 158, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-22T01:03:24.442 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:03:24.443 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.75, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.75, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-22T01:03:24.444 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "8c0f94a8-3c63-442d-9dc1-50c1278929da", 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:24.445 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 146, 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.448 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-22T01:03:24.449 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2277223948 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2277223948", 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2277223948", 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2277223948", 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2277223948", 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:03:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "2d2cf30e-e9bd-40ba-8293-193faa02ea69", 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 142, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.451 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-22T01:03:24.452 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:24.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 463340194 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/463340194", 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/463340194", 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/463340194", 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/463340194", 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a3d3e330-9ea1-4a5c-811b-f285f9a3cff4", 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 143, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:24.456 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 387643397 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/387643397", 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/387643397", 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/387643397", 2024-09-22T01:03:24.460 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/387643397", 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "239072f0-4b69-4624-ba6c-733f299208da", 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:24.461 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 144, 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-22T01:03:24.462 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:24.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.464 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1215210764 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1215210764", 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1215210764", 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1215210764", 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1215210764", 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fd35cb24-eb2c-4c17-abe9-ce5c34db9014", 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:24.466 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 155, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6800", 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:24.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6801", 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6802", 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:24.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6803", 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6806", 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6807", 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6804", 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6805", 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2463696152 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6801/2463696152", 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6803/2463696152", 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6807/2463696152", 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6805/2463696152", 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "b60a584d-5334-41e4-92d9-f6689de6b8bf", 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 15, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 25, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 52, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 145, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 26, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:24.472 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6824", 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6825", 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6826", 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6827", 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6830", 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6831", 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6828", 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6829", 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1737475131 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6825/1737475131", 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6827/1737475131", 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1737475131", 2024-09-22T01:03:24.476 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1737475131", 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "9be24894-1100-4e4e-bf1b-79e163fd97e0", 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:24.477 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 146, 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6816", 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6817", 2024-09-22T01:03:24.478 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6818", 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6819", 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:24.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6822", 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6823", 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6820", 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6821", 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4220217922 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6817/4220217922", 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6819/4220217922", 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6823/4220217922", 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6821/4220217922", 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "293cfbc3-8c2e-40b5-ab89-0a124cc8651c", 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-22T01:03:24.482 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 156, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6808", 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:24.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6809", 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6810", 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6811", 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6814", 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6815", 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6812", 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.119:6813", 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3867156106 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.119:6809/3867156106", 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.119:6811/3867156106", 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3867156106", 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3867156106", 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.487 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.556128+0000", 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:24.488 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.159753+0000", 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:24.489 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.766199+0000", 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.155529+0000", 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-22T01:03:24.490 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.226232+0000", 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:24.491 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.243346+0000", 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 26 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:28.143084+0000", 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-22T01:00:27.524369+0000", 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.17", 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-22T01:03:24.493 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: 1 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [ 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "2.0", 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 1, 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 2 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.13", 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 7, 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 3 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-22T01:03:24.495 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [ 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.496 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 18, 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 9 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [ 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 18, 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 1 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.497 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:24.498 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:24.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:24.500 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:24.500 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:24.500 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:24.496+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:24.501 INFO:tasks.thrashosds.thrasher:j is {'epoch': 158, 'fsid': '7d08b428-4185-4f0f-a0fc-f7dc1fefdce9', 'created': '2024-09-22T01:00:18.015746+0000', 'modified': '2024-09-22T01:03:23.735725+0000', 'last_up_change': '2024-09-22T01:01:26.960327+0000', 'last_in_change': '2024-09-22T01:03:05.961739+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:00:30.699559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:00:36.410605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:00:46.342370+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 158, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8c0f94a8-3c63-442d-9dc1-50c1278929da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2277223948}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2277223948}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2277223948}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2277223948}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2277223948}]}, 'public_addr': '172.21.15.2:6825/2277223948', 'cluster_addr': '172.21.15.2:6827/2277223948', 'heartbeat_back_addr': '172.21.15.2:6831/2277223948', 'heartbeat_front_addr': '172.21.15.2:6829/2277223948', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2d2cf30e-e9bd-40ba-8293-193faa02ea69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 463340194}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 463340194}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 463340194}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 463340194}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 463340194}]}, 'public_addr': '172.21.15.2:6817/463340194', 'cluster_addr': '172.21.15.2:6819/463340194', 'heartbeat_back_addr': '172.21.15.2:6823/463340194', 'heartbeat_front_addr': '172.21.15.2:6821/463340194', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a3d3e330-9ea1-4a5c-811b-f285f9a3cff4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 387643397}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 387643397}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 387643397}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 387643397}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 387643397}]}, 'public_addr': '172.21.15.2:6801/387643397', 'cluster_addr': '172.21.15.2:6803/387643397', 'heartbeat_back_addr': '172.21.15.2:6807/387643397', 'heartbeat_front_addr': '172.21.15.2:6805/387643397', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '239072f0-4b69-4624-ba6c-733f299208da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1215210764}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1215210764}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1215210764}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1215210764}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1215210764}]}, 'public_addr': '172.21.15.2:6809/1215210764', 'cluster_addr': '172.21.15.2:6811/1215210764', 'heartbeat_back_addr': '172.21.15.2:6815/1215210764', 'heartbeat_front_addr': '172.21.15.2:6813/1215210764', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd35cb24-eb2c-4c17-abe9-ce5c34db9014', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 155, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 2463696152}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 2463696152}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 2463696152}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 2463696152}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 2463696152}]}, 'public_addr': '172.21.15.119:6801/2463696152', 'cluster_addr': '172.21.15.119:6803/2463696152', 'heartbeat_back_addr': '172.21.15.119:6807/2463696152', 'heartbeat_front_addr': '172.21.15.119:6805/2463696152', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b60a584d-5334-41e4-92d9-f6689de6b8bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 25, 'up_from': 52, 'up_thru': 145, 'down_at': 26, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1737475131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1737475131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1737475131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1737475131}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1737475131}]}, 'public_addr': '172.21.15.119:6825/1737475131', 'cluster_addr': '172.21.15.119:6827/1737475131', 'heartbeat_back_addr': '172.21.15.119:6831/1737475131', 'heartbeat_front_addr': '172.21.15.119:6829/1737475131', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9be24894-1100-4e4e-bf1b-79e163fd97e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 4220217922}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 4220217922}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 4220217922}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 4220217922}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 4220217922}]}, 'public_addr': '172.21.15.119:6817/4220217922', 'cluster_addr': '172.21.15.119:6819/4220217922', 'heartbeat_back_addr': '172.21.15.119:6823/4220217922', 'heartbeat_front_addr': '172.21.15.119:6821/4220217922', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '293cfbc3-8c2e-40b5-ab89-0a124cc8651c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 156, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3867156106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3867156106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3867156106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3867156106}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3867156106}]}, 'public_addr': '172.21.15.119:6809/3867156106', 'cluster_addr': '172.21.15.119:6811/3867156106', 'heartbeat_back_addr': '172.21.15.119:6815/3867156106', 'heartbeat_front_addr': '172.21.15.119:6813/3867156106', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.556128+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.159753+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.766199+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.155529+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.226232+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.243346+0000', 'dead_epoch': 26}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:28.143084+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:00:27.524369+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.17', 'osds': [6, 1]}], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '3.13', 'mappings': [{'from': 7, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 18, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 18, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:03:24.502 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:03:24.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:24.600 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:24.596+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:24.670 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:24.670 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:24.685 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:24.703 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:24.703 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:24.720 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:24.720 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:24.729 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:24.728+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:24.738 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:24.738 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:24.762 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:24.762 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:24.769 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:03:24.788 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:24.789 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:24.803 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:24.800+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:24.809 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:24.809 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:24.812 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:24.812 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:24.828 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:24.845 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:24.845 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:24.867 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:24.891 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:03:24.892 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:03:24.903 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:24.900+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:24.911 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-22T01:03:24.743496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461136995,"num_objects":798,"num_object_clones":183,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":414,"num_read":198272,"num_read_kb":756200,"num_write":21960,"num_write_kb":413304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4364,"num_bytes_recovered":210023584,"num_keys_recovered":916,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18790,"ondisk_log_size":18790,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":484040,"kb_used_data":139024,"kb_used_omap":15,"kb_used_meta":344816,"kb_avail":754490680,"statfs":{"total":773094113280,"available":772598456320,"internally_reserved":0,"allocated":142360576,"data_stored":158893187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":353092281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8995937,"num_objects":-11,"num_object_clones":-4,"num_object_copies":-22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-11,"num_whiteouts":-4,"num_read":341,"num_read_kb":297,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.664877"},"pg_stats":[{"pgid":"3.1f","version":"154'499","reported_seq":6983,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.094584+0000","last_change":"2024-09-22T01:03:20.095026+0000","last_active":"2024-09-22T01:03:20.094584+0000","last_peered":"2024-09-22T01:03:20.094584+0000","last_clean":"2024-09-22T01:03:20.094584+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:03:20.094584+0000","last_undegraded":"2024-09-22T01:03:20.094584+0000","last_fullsized":"2024-09-22T01:03:20.094584+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:46:23.542774+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0098235019999999996,"stat_sum":{"num_bytes":23109569,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":6502,"num_read_kb":45938,"num_write":531,"num_write_kb":17793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.1e","version":"154'732","reported_seq":3403,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.083929+0000","last_change":"2024-09-22T01:03:20.084273+0000","last_active":"2024-09-22T01:03:20.083929+0000","last_peered":"2024-09-22T01:03:20.083929+0000","last_clean":"2024-09-22T01:03:20.083929+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:03:20.083929+0000","last_undegraded":"2024-09-22T01:03:20.083929+0000","last_fullsized":"2024-09-22T01:03:20.083929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:50:25.243944+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0065305850000000002,"stat_sum":{"num_bytes":9980232,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":6094,"num_read_kb":12764,"num_write":941,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.1d","version":"150'457","reported_seq":5015,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.093150+0000","last_change":"2024-09-22T01:03:20.093383+0000","last_active":"2024-09-22T01:03:20.093150+0000","last_peered":"2024-09-22T01:03:20.093150+0000","last_clean":"2024-09-22T01:03:20.093150+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:03:20.093150+0000","last_undegraded":"2024-09-22T01:03:20.093150+0000","last_fullsized":"2024-09-22T01:03:20.093150+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:19.263090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401183,"stat_sum":{"num_bytes":15761467,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":4787,"num_read_kb":32328,"num_write":476,"num_write_kb":18066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.1c","version":"154'1072","reported_seq":10404,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.204204+0000","last_change":"2024-09-22T01:03:20.094963+0000","last_active":"2024-09-22T01:03:20.204204+0000","last_peered":"2024-09-22T01:03:20.204204+0000","last_clean":"2024-09-22T01:03:20.204204+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:03:20.204204+0000","last_undegraded":"2024-09-22T01:03:20.204204+0000","last_fullsized":"2024-09-22T01:03:20.204204+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1072,"log_dups_size":0,"ondisk_log_size":1072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:39:10.178649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096072599999999996,"stat_sum":{"num_bytes":30558490,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":11362,"num_read_kb":51602,"num_write":1550,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.1b","version":"154'1074","reported_seq":2939,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.090193+0000","last_change":"2024-09-22T01:03:20.090193+0000","last_active":"2024-09-22T01:03:20.090193+0000","last_peered":"2024-09-22T01:03:20.090193+0000","last_clean":"2024-09-22T01:03:20.090193+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:03:20.090193+0000","last_undegraded":"2024-09-22T01:03:20.090193+0000","last_fullsized":"2024-09-22T01:03:20.090193+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:29:47.738477+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051183419999999997,"stat_sum":{"num_bytes":9607427,"num_objects":27,"num_object_clones":11,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":14,"num_read":5211,"num_read_kb":15422,"num_write":1520,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.1a","version":"148'467","reported_seq":12057,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.219871+0000","last_change":"2024-09-22T01:03:20.074563+0000","last_active":"2024-09-22T01:03:20.219871+0000","last_peered":"2024-09-22T01:03:20.219871+0000","last_clean":"2024-09-22T01:03:20.219871+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:03:20.219871+0000","last_undegraded":"2024-09-22T01:03:20.219871+0000","last_fullsized":"2024-09-22T01:03:20.219871+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:03:40.265229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015664159999999999,"stat_sum":{"num_bytes":20203244,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":12036,"num_read_kb":25878,"num_write":539,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.19","version":"154'388","reported_seq":9892,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.076414+0000","last_change":"2024-09-22T01:03:20.076414+0000","last_active":"2024-09-22T01:03:20.076414+0000","last_peered":"2024-09-22T01:03:20.076414+0000","last_clean":"2024-09-22T01:03:20.076414+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:03:20.076414+0000","last_undegraded":"2024-09-22T01:03:20.076414+0000","last_fullsized":"2024-09-22T01:03:20.076414+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:33:57.006844+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0035307260000000001,"stat_sum":{"num_bytes":13866134,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":9132,"num_read_kb":31302,"num_write":449,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.18","version":"153'742","reported_seq":6324,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.075675+0000","last_change":"2024-09-22T01:03:20.076291+0000","last_active":"2024-09-22T01:03:20.075675+0000","last_peered":"2024-09-22T01:03:20.075675+0000","last_clean":"2024-09-22T01:03:20.075675+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:03:20.075675+0000","last_undegraded":"2024-09-22T01:03:20.075675+0000","last_fullsized":"2024-09-22T01:03:20.075675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:29.633498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001027214,"stat_sum":{"num_bytes":20088508,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":8378,"num_read_kb":42523,"num_write":815,"num_write_kb":16840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.17","version":"152'550","reported_seq":4221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-22T01:03:21.086322+0000","last_change":"2024-09-22T01:03:21.087216+0000","last_active":"2024-09-22T01:03:21.086322+0000","last_peered":"2024-09-22T01:03:21.086322+0000","last_clean":"2024-09-22T01:03:21.086322+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:03:21.086322+0000","last_undegraded":"2024-09-22T01:03:21.086322+0000","last_fullsized":"2024-09-22T01:03:21.086322+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:33:24.238109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020220250000000002,"stat_sum":{"num_bytes":5637929,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":12,"num_read":4907,"num_read_kb":18393,"num_write":625,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"15","length":"a"}]},{"pgid":"3.16","version":"150'350","reported_seq":2077,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.093354+0000","last_change":"2024-09-22T01:03:20.093354+0000","last_active":"2024-09-22T01:03:20.093354+0000","last_peered":"2024-09-22T01:03:20.093354+0000","last_clean":"2024-09-22T01:03:20.093354+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:03:20.093354+0000","last_undegraded":"2024-09-22T01:03:20.093354+0000","last_fullsized":"2024-09-22T01:03:20.093354+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:31.643481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400832,"stat_sum":{"num_bytes":10723503,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":2583,"num_read_kb":34286,"num_write":386,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.15","version":"154'626","reported_seq":6541,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.084356+0000","last_change":"2024-09-22T01:03:20.085644+0000","last_active":"2024-09-22T01:03:20.084356+0000","last_peered":"2024-09-22T01:03:20.084356+0000","last_clean":"2024-09-22T01:03:20.084356+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:03:20.084356+0000","last_undegraded":"2024-09-22T01:03:20.084356+0000","last_fullsized":"2024-09-22T01:03:20.084356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:22:12.439327+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0076398600000000001,"stat_sum":{"num_bytes":19495845,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":14,"num_read":8608,"num_read_kb":26892,"num_write":651,"num_write_kb":5755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.14","version":"154'516","reported_seq":13746,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.183842+0000","last_change":"2024-09-22T01:03:20.077249+0000","last_active":"2024-09-22T01:03:20.183842+0000","last_peered":"2024-09-22T01:03:20.183842+0000","last_clean":"2024-09-22T01:03:20.183842+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:03:20.183842+0000","last_undegraded":"2024-09-22T01:03:20.183842+0000","last_fullsized":"2024-09-22T01:03:20.183842+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:53:33.316149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038005729999999998,"stat_sum":{"num_bytes":23355301,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":13844,"num_read_kb":40938,"num_write":535,"num_write_kb":17925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":288,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-22T01:03:21.085395+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:03:21.085395+0000","last_peered":"2024-09-22T01:03:21.085395+0000","last_clean":"2024-09-22T01:03:21.085395+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:03:21.085395+0000","last_undegraded":"2024-09-22T01:03:21.085395+0000","last_fullsized":"2024-09-22T01:03:21.085395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"154'536","reported_seq":2938,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.234548+0000","last_change":"2024-09-22T01:03:20.090962+0000","last_active":"2024-09-22T01:03:20.234548+0000","last_peered":"2024-09-22T01:03:20.234548+0000","last_clean":"2024-09-22T01:03:20.234548+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:03:20.234548+0000","last_undegraded":"2024-09-22T01:03:20.234548+0000","last_fullsized":"2024-09-22T01:03:20.234548+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:26:57.086006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058227249999999999,"stat_sum":{"num_bytes":9271343,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":4023,"num_read_kb":6453,"num_write":525,"num_write_kb":3324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":100,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.073316+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:03:20.073316+0000","last_peered":"2024-09-22T01:03:20.073316+0000","last_clean":"2024-09-22T01:03:20.073316+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:03:20.073316+0000","last_undegraded":"2024-09-22T01:03:20.073316+0000","last_fullsized":"2024-09-22T01:03:20.073316+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'539","reported_seq":3440,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.077875+0000","last_change":"2024-09-22T01:03:20.078497+0000","last_active":"2024-09-22T01:03:20.077875+0000","last_peered":"2024-09-22T01:03:20.077875+0000","last_clean":"2024-09-22T01:03:20.077875+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:03:20.077875+0000","last_undegraded":"2024-09-22T01:03:20.077875+0000","last_fullsized":"2024-09-22T01:03:20.077875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:06.402962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554918,"stat_sum":{"num_bytes":10894425,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":4463,"num_read_kb":12695,"num_write":674,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":286,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.079019+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:03:20.079019+0000","last_peered":"2024-09-22T01:03:20.079019+0000","last_clean":"2024-09-22T01:03:20.079019+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:03:20.079019+0000","last_undegraded":"2024-09-22T01:03:20.079019+0000","last_fullsized":"2024-09-22T01:03:20.079019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"153'429","reported_seq":4319,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.075537+0000","last_change":"2024-09-22T01:03:20.076019+0000","last_active":"2024-09-22T01:03:20.075537+0000","last_peered":"2024-09-22T01:03:20.075537+0000","last_clean":"2024-09-22T01:03:20.075537+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:03:20.075537+0000","last_undegraded":"2024-09-22T01:03:20.075537+0000","last_fullsized":"2024-09-22T01:03:20.075537+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:53:36.623855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002947009,"stat_sum":{"num_bytes":8090593,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":15,"num_read":4002,"num_read_kb":26479,"num_write":407,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":100,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.072867+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:03:20.072867+0000","last_peered":"2024-09-22T01:03:20.072867+0000","last_clean":"2024-09-22T01:03:20.072867+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:03:20.072867+0000","last_undegraded":"2024-09-22T01:03:20.072867+0000","last_fullsized":"2024-09-22T01:03:20.072867+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"154'580","reported_seq":4050,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.095848+0000","last_change":"2024-09-22T01:03:20.096529+0000","last_active":"2024-09-22T01:03:20.095848+0000","last_peered":"2024-09-22T01:03:20.095848+0000","last_clean":"2024-09-22T01:03:20.095848+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:03:20.095848+0000","last_undegraded":"2024-09-22T01:03:20.095848+0000","last_fullsized":"2024-09-22T01:03:20.095848+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:37:02.617334+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033142390000000001,"stat_sum":{"num_bytes":8332600,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":6331,"num_read_kb":12165,"num_write":515,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.3","version":"154'629","reported_seq":2725,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.192246+0000","last_change":"2024-09-22T01:03:20.075534+0000","last_active":"2024-09-22T01:03:20.192246+0000","last_peered":"2024-09-22T01:03:20.192246+0000","last_clean":"2024-09-22T01:03:20.192246+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:03:20.192246+0000","last_undegraded":"2024-09-22T01:03:20.192246+0000","last_fullsized":"2024-09-22T01:03:20.192246+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:55:47.926578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021740190000000001,"stat_sum":{"num_bytes":11063974,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":3260,"num_read_kb":17877,"num_write":662,"num_write_kb":15166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"2.2","version":"21'2","reported_seq":258,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.092782+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:03:20.092782+0000","last_peered":"2024-09-22T01:03:20.092782+0000","last_clean":"2024-09-22T01:03:20.092782+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:03:20.092782+0000","last_undegraded":"2024-09-22T01:03:20.092782+0000","last_fullsized":"2024-09-22T01:03:20.092782+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"152'1611","reported_seq":3648,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.073135+0000","last_change":"2024-09-22T01:03:20.075155+0000","last_active":"2024-09-22T01:03:20.073135+0000","last_peered":"2024-09-22T01:03:20.073135+0000","last_clean":"2024-09-22T01:03:20.073135+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:03:20.073135+0000","last_undegraded":"2024-09-22T01:03:20.073135+0000","last_fullsized":"2024-09-22T01:03:20.073135+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1611,"log_dups_size":0,"ondisk_log_size":1611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:14:50.347357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022191670000000002,"stat_sum":{"num_bytes":14804018,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":8932,"num_read_kb":25333,"num_write":2678,"num_write_kb":16906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":286,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.072162+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:03:20.072162+0000","last_peered":"2024-09-22T01:03:20.072162+0000","last_clean":"2024-09-22T01:03:20.072162+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:03:20.072162+0000","last_undegraded":"2024-09-22T01:03:20.072162+0000","last_fullsized":"2024-09-22T01:03:20.072162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"154'365","reported_seq":2959,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.232724+0000","last_change":"2024-09-22T01:03:20.086777+0000","last_active":"2024-09-22T01:03:20.232724+0000","last_peered":"2024-09-22T01:03:20.232724+0000","last_clean":"2024-09-22T01:03:20.232724+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:03:20.232724+0000","last_undegraded":"2024-09-22T01:03:20.232724+0000","last_fullsized":"2024-09-22T01:03:20.232724+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:00:06.962926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020364950000000001,"stat_sum":{"num_bytes":7061004,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":2197,"num_read_kb":18585,"num_write":380,"num_write_kb":15217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":301,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.077379+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:03:20.077379+0000","last_peered":"2024-09-22T01:03:20.077379+0000","last_clean":"2024-09-22T01:03:20.077379+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:03:20.077379+0000","last_undegraded":"2024-09-22T01:03:20.077379+0000","last_fullsized":"2024-09-22T01:03:20.077379+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"153'356","reported_seq":2174,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.076975+0000","last_change":"2024-09-22T01:03:20.077426+0000","last_active":"2024-09-22T01:03:20.076975+0000","last_peered":"2024-09-22T01:03:20.076975+0000","last_clean":"2024-09-22T01:03:20.076975+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:03:20.076975+0000","last_undegraded":"2024-09-22T01:03:20.076975+0000","last_fullsized":"2024-09-22T01:03:20.076975+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:31:15.896097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063016700000000005,"stat_sum":{"num_bytes":13025735,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":1621,"num_read_kb":15577,"num_write":356,"num_write_kb":17862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"2.3","version":"19'1","reported_seq":239,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.092875+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:03:20.092875+0000","last_peered":"2024-09-22T01:03:20.092875+0000","last_clean":"2024-09-22T01:03:20.092875+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:03:20.092875+0000","last_undegraded":"2024-09-22T01:03:20.092875+0000","last_fullsized":"2024-09-22T01:03:20.092875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":437,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.077502+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:03:20.077502+0000","last_peered":"2024-09-22T01:03:20.077502+0000","last_clean":"2024-09-22T01:03:20.077502+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:03:20.077502+0000","last_undegraded":"2024-09-22T01:03:20.077502+0000","last_fullsized":"2024-09-22T01:03:20.077502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"153'427","reported_seq":10387,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.195254+0000","last_change":"2024-09-22T01:03:20.077064+0000","last_active":"2024-09-22T01:03:20.195254+0000","last_peered":"2024-09-22T01:03:20.195254+0000","last_clean":"2024-09-22T01:03:20.195254+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:03:20.195254+0000","last_undegraded":"2024-09-22T01:03:20.195254+0000","last_fullsized":"2024-09-22T01:03:20.195254+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:01:08.215116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014547869999999999,"stat_sum":{"num_bytes":10144999,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":10551,"num_read_kb":27262,"num_write":414,"num_write_kb":13697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.9","version":"152'571","reported_seq":2301,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.073361+0000","last_change":"2024-09-22T01:03:20.074215+0000","last_active":"2024-09-22T01:03:20.073361+0000","last_peered":"2024-09-22T01:03:20.073361+0000","last_clean":"2024-09-22T01:03:20.073361+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:03:20.073361+0000","last_undegraded":"2024-09-22T01:03:20.073361+0000","last_fullsized":"2024-09-22T01:03:20.073361+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:45:02.764931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011456940000000001,"stat_sum":{"num_bytes":20508189,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":13,"num_read":2831,"num_read_kb":7165,"num_write":642,"num_write_kb":8025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.a","version":"155'763","reported_seq":16002,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-22T01:03:21.086212+0000","last_change":"2024-09-22T01:03:21.086212+0000","last_active":"2024-09-22T01:03:21.086212+0000","last_peered":"2024-09-22T01:03:21.086212+0000","last_clean":"2024-09-22T01:03:21.086212+0000","last_became_active":"2024-09-22T01:03:12.750106+0000","last_became_peered":"2024-09-22T01:03:12.750106+0000","last_unstale":"2024-09-22T01:03:21.086212+0000","last_undegraded":"2024-09-22T01:03:21.086212+0000","last_fullsized":"2024-09-22T01:03:21.086212+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:42:56.510473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001282921,"stat_sum":{"num_bytes":8134055,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":16591,"num_read_kb":40539,"num_write":790,"num_write_kb":13037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":13815709,"num_keys_recovered":200,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"15","length":"a"}]},{"pgid":"3.b","version":"154'630","reported_seq":2550,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.235225+0000","last_change":"2024-09-22T01:03:20.078536+0000","last_active":"2024-09-22T01:03:20.235225+0000","last_peered":"2024-09-22T01:03:20.235225+0000","last_clean":"2024-09-22T01:03:20.235225+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:03:20.235225+0000","last_undegraded":"2024-09-22T01:03:20.235225+0000","last_fullsized":"2024-09-22T01:03:20.235225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:53.720227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001431559,"stat_sum":{"num_bytes":6715977,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":14,"num_read":4044,"num_read_kb":19628,"num_write":768,"num_write_kb":10854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.c","version":"153'381","reported_seq":1296,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.213636+0000","last_change":"2024-09-22T01:03:20.095338+0000","last_active":"2024-09-22T01:03:20.213636+0000","last_peered":"2024-09-22T01:03:20.213636+0000","last_clean":"2024-09-22T01:03:20.213636+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:03:20.213636+0000","last_undegraded":"2024-09-22T01:03:20.213636+0000","last_fullsized":"2024-09-22T01:03:20.213636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:44.834603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005234,"stat_sum":{"num_bytes":16933872,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":2635,"num_read_kb":21720,"num_write":378,"num_write_kb":11071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.d","version":"150'738","reported_seq":5823,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.079741+0000","last_change":"2024-09-22T01:03:20.081514+0000","last_active":"2024-09-22T01:03:20.079741+0000","last_peered":"2024-09-22T01:03:20.079741+0000","last_clean":"2024-09-22T01:03:20.079741+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:03:20.079741+0000","last_undegraded":"2024-09-22T01:03:20.079741+0000","last_fullsized":"2024-09-22T01:03:20.079741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:09:38.579479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028593239999999999,"stat_sum":{"num_bytes":18341888,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":15,"num_read":7435,"num_read_kb":29071,"num_write":888,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.e","version":"153'453","reported_seq":4992,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.086167+0000","last_change":"2024-09-22T01:03:20.086679+0000","last_active":"2024-09-22T01:03:20.086167+0000","last_peered":"2024-09-22T01:03:20.086167+0000","last_clean":"2024-09-22T01:03:20.086167+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:03:20.086167+0000","last_undegraded":"2024-09-22T01:03:20.086167+0000","last_fullsized":"2024-09-22T01:03:20.086167+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:08:55.137730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072959419999999997,"stat_sum":{"num_bytes":32452765,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":3826,"num_read_kb":26267,"num_write":502,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.f","version":"154'368","reported_seq":4917,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.086430+0000","last_change":"2024-09-22T01:03:20.086991+0000","last_active":"2024-09-22T01:03:20.086430+0000","last_peered":"2024-09-22T01:03:20.086430+0000","last_clean":"2024-09-22T01:03:20.086430+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:03:20.086430+0000","last_undegraded":"2024-09-22T01:03:20.086430+0000","last_fullsized":"2024-09-22T01:03:20.086430+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:22:29.896583+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080390689999999994,"stat_sum":{"num_bytes":8978448,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":9,"num_read":5001,"num_read_kb":10198,"num_write":294,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.10","version":"155'446","reported_seq":3219,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-22T01:03:21.090881+0000","last_change":"2024-09-22T01:03:21.091333+0000","last_active":"2024-09-22T01:03:21.090881+0000","last_peered":"2024-09-22T01:03:21.090881+0000","last_clean":"2024-09-22T01:03:21.090881+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:03:21.090881+0000","last_undegraded":"2024-09-22T01:03:21.090881+0000","last_fullsized":"2024-09-22T01:03:21.090881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:11:42.089780+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0064636190000000003,"stat_sum":{"num_bytes":17628607,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":16,"num_read":3238,"num_read_kb":14715,"num_write":388,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"15","length":"a"}]},{"pgid":"3.11","version":"144'642","reported_seq":2827,"reported_epoch":152,"state":"remapped+peering","last_fresh":"2024-09-22T01:03:18.056636+0000","last_change":"2024-09-22T01:03:20.073373+0000","last_active":"2024-09-22T01:03:20.073373+0000","last_peered":"2024-09-22T01:03:18.056636+0000","last_clean":"2024-09-22T01:03:18.056636+0000","last_became_active":"2024-09-22T01:03:09.761341+0000","last_became_peered":"2024-09-22T01:03:09.761341+0000","last_unstale":"2024-09-22T01:03:18.056636+0000","last_undegraded":"2024-09-22T01:03:18.056636+0000","last_fullsized":"2024-09-22T01:03:18.056636+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015276910000000001,"stat_sum":{"num_bytes":15555596,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":4508,"num_read_kb":11727,"num_write":781,"num_write_kb":5789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2273258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":7,"acting_primary":4,"purged_snaps":[{"start":"18","length":"7"}]},{"pgid":"3.12","version":"154'429","reported_seq":3072,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.089907+0000","last_change":"2024-09-22T01:03:20.090277+0000","last_active":"2024-09-22T01:03:20.089907+0000","last_peered":"2024-09-22T01:03:20.089907+0000","last_clean":"2024-09-22T01:03:20.089907+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:03:20.089907+0000","last_undegraded":"2024-09-22T01:03:20.089907+0000","last_fullsized":"2024-09-22T01:03:20.089907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:58:14.871565+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0055033770000000003,"stat_sum":{"num_bytes":8612205,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":16,"num_read":3530,"num_read_kb":7508,"num_write":403,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"9"}]},{"pgid":"3.13","version":"150'304","reported_seq":6705,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-22T01:03:20.077122+0000","last_change":"2024-09-22T01:03:20.077841+0000","last_active":"2024-09-22T01:03:20.077122+0000","last_peered":"2024-09-22T01:03:20.077122+0000","last_clean":"2024-09-22T01:03:20.077122+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:03:20.077122+0000","last_undegraded":"2024-09-22T01:03:20.077122+0000","last_fullsized":"2024-09-22T01:03:20.077122+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:03:43.550994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096323000000000003,"stat_sum":{"num_bytes":11608666,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":14,"num_read":5693,"num_read_kb":26870,"num_write":336,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":460546608,"num_objects":794,"num_object_clones":183,"num_object_copies":1588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":794,"num_whiteouts":414,"num_read":198156,"num_read_kb":756100,"num_write":21799,"num_write_kb":410886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4362,"num_bytes_recovered":210023546,"num_keys_recovered":916,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":126410752,"data_stored":150639408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":18670,"ondisk_log_size":18670,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38248,"kb_used_data":7060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333592,"statfs":{"total":96636764160,"available":96597598208,"internally_reserved":0,"allocated":7229440,"data_stored":7485770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299416,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57028,"kb_used_data":28904,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94314812,"statfs":{"total":96636764160,"available":96578367488,"internally_reserved":0,"allocated":29597696,"data_stored":35738029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75388,"kb_used_data":17832,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94296452,"statfs":{"total":96636764160,"available":96559566848,"internally_reserved":0,"allocated":18259968,"data_stored":21528705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55692,"kb_used_data":24500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316148,"statfs":{"total":96636764160,"available":96579735552,"internally_reserved":0,"allocated":25088000,"data_stored":27839543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82336,"kb_used_data":24776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94289504,"statfs":{"total":96636764160,"available":96552452096,"internally_reserved":0,"allocated":25370624,"data_stored":26300973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54444,"kb_used_data":5208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317396,"statfs":{"total":96636764160,"available":96581013504,"internally_reserved":0,"allocated":5332992,"data_stored":6563438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53224,"kb_used_data":8140,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318616,"statfs":{"total":96636764160,"available":96582262784,"internally_reserved":0,"allocated":8335360,"data_stored":9694099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542182,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67680,"kb_used_data":22604,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94304160,"statfs":{"total":96636764160,"available":96567459840,"internally_reserved":0,"allocated":23146496,"data_stored":23742630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20709376,"data_stored":22268140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6488064,"data_stored":8809958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3502080,"data_stored":5685656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23519232,"data_stored":25416797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23244800,"data_stored":26955421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27750400,"data_stored":34853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5373952,"data_stored":6595333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15822848,"data_stored":20054215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:03:24.913 INFO:tasks.thrashosds.thrasher:Setting 3.a to [2, 3, 4, 1] 2024-09-22T01:03:24.913 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.a', '2', '3', '4', '1'] 2024-09-22T01:03:24.914 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.a 2 3 4 1 2024-09-22T01:03:24.997 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:24.998 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:25.004 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:25.000+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:25.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:25.105 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:25.100+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:25.174 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:25.174 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:25.174 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:25.174 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:25.174 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:25.188 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:25.233 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:25.232+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:25.307 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:25.304+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:25.308 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:25.308 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:25.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:25.308 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:25.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.308 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 173.41455725399999, 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:03:25.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:03:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:03:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:03:25.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.030559927, 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:25.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.030526622, 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.319 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.03043010899999, 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.030356858, 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.327 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:25.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:03:25.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.03029167699999, 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:03:25.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:03:25.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.030221556, 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:25.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.03018819099999, 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.339 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.030157769, 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.343 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.030127463, 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:03:25.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.03009706100001, 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.03006626200001, 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:03:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:03:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.030035789, 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.03000519699999, 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.029967395, 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.02993503, 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.029904711, 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.029874819, 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.02984506, 2024-09-22T01:03:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.029794595, 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:03:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:03:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.390 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:25.391 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:25.433 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:25.432+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:25.508 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:25.504+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:25.558 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:25.559 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:25.559 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:25.559 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:25.573 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:25.617 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:25.616+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:25.711 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:25.708+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:25.742 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:25.743 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:25.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:set 3.a pg_upmap_items mapping to [2->3,4->1] 2024-09-22T01:03:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:03:25.811 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:25.808+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:25.887 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756909725, 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:03:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.75685405, 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756801705, 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:03:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756743794, 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:03:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:03:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:03:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756685155, 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:03:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.75662992300001, 2024-09-22T01:03:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:03:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:03:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:03:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.75657619899999, 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:03:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756504464, 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:03:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:03:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756447083, 2024-09-22T01:03:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:03:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756396817, 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:03:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.75631776, 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:03:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:03:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:03:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.75619351100001, 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:03:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:03:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756112929, 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.756047741, 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:03:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.755272772, 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:03:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:03:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.755222987, 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:03:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.755141886, 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.75507190099999, 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:03:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:03:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.754994537, 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:03:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.75493401599999, 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:03:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:03:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:03:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:25.989 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:25.991 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:25.988+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:25.991 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:26.092 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:26.088+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:26.127 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:26.127 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:26.127 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:26.127 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:26.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:26.238 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:26.236+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:26.294 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:26.288+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:26.318 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:26.318 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:26.318 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:26.318 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:26.318 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:26.332 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:26.395 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:26.392+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:26.496 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:26.492+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:26.498 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:26.498 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:26.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:26.498 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:26.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.9963121, 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:03:26.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:03:26.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.996284257, 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:03:26.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:26.507 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.99625204100001, 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:03:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:03:26.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.996230303, 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:03:26.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:03:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.996210648, 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 104.996179634, 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:26.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040775422999999, 2024-09-22T01:03:26.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:03:26.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.528 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040735788000006, 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:26.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040648347000001, 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:03:26.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:03:26.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:03:26.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040608128000002, 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:03:26.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040572236000003, 2024-09-22T01:03:26.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:03:26.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:03:26.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040524587999997, 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.549 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040489819000001, 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040456625999994, 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040412411999995, 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:03:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040370222999996, 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:03:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:03:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040336111000002, 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:03:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040303183999995, 2024-09-22T01:03:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:03:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:03:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.040268416000004, 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:03:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:151264 3.1b 3:deb34d03:::rbd_object_map.117cf172dcd2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.502079663, 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.812141679, 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 151264 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993280+0000", 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8669999999999998e-06 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993295+0000", 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5235999999999999e-05 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993321+0000", 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5775e-05 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993335+0000", 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4306e-05 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993415+0000", 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9975999999999995e-05 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993426+0000", 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1063000000000001e-05 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978616+0000", 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98518988299999999 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978652+0000", 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5976000000000002e-05 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.803938+0000", 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.82528633799999995 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804052+0000", 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113491 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804224+0000", 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172375 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805210+0000", 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000986385 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805298+0000", 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7410000000000005e-05 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805343+0000", 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5451999999999998e-05 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805418+0000", 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5146000000000002e-05 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:26.587 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:26.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:26.638 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:26.636+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:26.697 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:26.696+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:26.719 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:26.720 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:26.720 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:26.720 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:26.734 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:26.822 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:26.820+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:26.890 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:26.898 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:26.899 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:26.899 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:26.896+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:26.914 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:27.006 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:27.004+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:27.033 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:27.034 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:27.050 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:27.051 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:27.070 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:27.089 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:27.089 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:27.092 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:27.093 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:27.101 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:27.100+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:27.107 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:27.111 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:27.133 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:27.133 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:27.155 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:27.178 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:27.178 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:27.202 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:27.200+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:27.232 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:27.233 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:27.247 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:27.334 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:27.332+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:27.404 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:27.400+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:27.416 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:27.416 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:27.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:27.526 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:27.524+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:27.606 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:27.606 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:27.606 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:27.604+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:27.620 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:27.707 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:27.704+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:27.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:03:27.803 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:27.803 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:27.808 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:27.804+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:27.821 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:27.910 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:27.908+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:27.997 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:27.997 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:28.011 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:28.008+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:28.013 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:28.113 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:28.108+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:28.172 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:28.172 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:28.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:28.213 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:28.208+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:28.314 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:28.312+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:28.318 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:28.318 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:28.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:28.421 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:28.420+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:28.502 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:28.502 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:28.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:28.516 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:28.512+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:28.616 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:28.612+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:28.681 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:28.682 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:28.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:28.717 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:28.712+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:28.777 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:03:28.818 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:28.816+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:28.870 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:28.870 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:28.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:28.919 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:28.916+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:29.020 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:29.016+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:29.050 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:29.050 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:29.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:29.121 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:29.116+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:29.197 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:29.231 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:29.232 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:29.246 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:29.246 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:29.244+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:29.322 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:29.320+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:29.353 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:29.353 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:29.366 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:29.367 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:29.369 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:29.369 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:29.381 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:29.387 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:29.387 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:29.403 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:29.403 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:29.421 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:29.422 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:29.429 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:29.428+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:29.439 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:29.440 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:29.458 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:29.459 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:29.479 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:29.479 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:29.538 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:29.536+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:29.554 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:29.554 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:29.554 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:29.554 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:29.554 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:29.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:29.625 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:29.624+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:29.691 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 177.79904531099999, 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.61078069400000001, 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4198", 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/2232135440", 2024-09-22T01:03:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891215+0000", 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891217+0000", 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3939999999999999e-06 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891231+0000", 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3672999999999999e-05 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891254+0000", 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2903000000000001e-05 2024-09-22T01:03:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891272+0000", 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8603000000000001e-05 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891394+0000", 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121996 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:31.891412+0000", 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.749e-05 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.498754+0000", 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.60734224999999997 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499083+0000", 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000328668 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.499447+0000", 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036413999999999999 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.500716+0000", 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012686799999999999 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501834+0000", 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001118546 2024-09-22T01:03:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501928+0000", 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4148999999999996e-05 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:00:32.501996+0000", 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7201999999999998e-05 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40979 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.415047984, 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147922785, 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40979 2024-09-22T01:03:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275212+0000", 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275214+0000", 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.313e-06 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275217+0000", 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6830000000000001e-06 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275227+0000", 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0223e-05 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275235+0000", 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8399999999999995e-06 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423112+0000", 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14787640699999999 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423135+0000", 2024-09-22T01:03:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319000000000002e-05 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40980 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.415014679, 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792493700000001, 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40980 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275246+0000", 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2699999999999998e-07 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275286+0000", 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9292000000000003e-05 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275299+0000", 2024-09-22T01:03:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275308+0000", 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5229999999999995e-06 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423161+0000", 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14785299599999999 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423170+0000", 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40985 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.41491816600001, 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792519300000001, 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40985 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275342+0000", 2024-09-22T01:03:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275343+0000", 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275347+0000", 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6459999999999999e-06 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275363+0000", 2024-09-22T01:03:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6269e-05 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275374+0000", 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0947e-05 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423240+0000", 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.147865841 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423267+0000", 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7455999999999999e-05 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40989 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414844915, 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795180099999999, 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40989 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275415+0000", 2024-09-22T01:03:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275416+0000", 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275419+0000", 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.326e-06 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275429+0000", 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3470000000000005e-06 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275436+0000", 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6329999999999996e-06 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423347+0000", 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14791029 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423367+0000", 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0565e-05 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40991 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.41477973400001, 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14792303400000001, 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40991 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275480+0000", 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275481+0000", 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.47e-07 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275485+0000", 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4060000000000001e-06 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275495+0000", 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0423e-05 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275503+0000", 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423393+0000", 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788985800000001 2024-09-22T01:03:29.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423403+0000", 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.076e-05 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40992 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414709613, 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14793566499999999, 2024-09-22T01:03:29.719 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40992 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275551+0000", 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275552+0000", 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.017e-06 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275555+0000", 2024-09-22T01:03:29.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275566+0000", 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0506000000000001e-05 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275574+0000", 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.824e-06 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423459+0000", 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14788536299999999 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423486+0000", 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7379e-05 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40993 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.41467624800001, 2024-09-22T01:03:29.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14794017100000001, 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40993 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275584+0000", 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275585+0000", 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275588+0000", 2024-09-22T01:03:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1669999999999999e-06 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275597+0000", 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275605+0000", 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5939999999999996e-06 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423514+0000", 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14790893399999999 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423524+0000", 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.029e-05 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40994 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414645826, 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14800619700000001, 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40994 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275614+0000", 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275615+0000", 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275618+0000", 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275628+0000", 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3940000000000002e-06 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275635+0000", 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3490000000000003e-06 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423591+0000", 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14795620500000001 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423621+0000", 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9428999999999999e-05 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40995 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.41461552, 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14805908700000001, 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40995 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275645+0000", 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275649+0000", 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275658+0000", 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3340000000000008e-06 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275665+0000", 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3560000000000004e-06 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423689+0000", 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14802346799999999 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423704+0000", 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4826e-05 2024-09-22T01:03:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40996 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.41458511800001, 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148190873, 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40996 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275675+0000", 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275676+0000", 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275679+0000", 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.067e-06 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275688+0000", 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999996e-06 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275696+0000", 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7659999999999998e-06 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423782+0000", 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148085943 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423866+0000", 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3897e-05 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40997 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414554319, 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820034500000001, 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40997 2024-09-22T01:03:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275706+0000", 2024-09-22T01:03:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1699999999999998e-07 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275710+0000", 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1700000000000001e-06 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275719+0000", 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2960000000000004e-06 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275726+0000", 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5329999999999997e-06 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423895+0000", 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148168298 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423906+0000", 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1430999999999999e-05 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40998 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.41452384599999, 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14825082000000001, 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40998 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275736+0000", 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275737+0000", 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275740+0000", 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275749+0000", 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3090000000000001e-06 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275757+0000", 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423959+0000", 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14820144599999999 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.423987+0000", 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8642000000000001e-05 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:40999 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.41449325400001, 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14826260399999999, 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 40999 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275767+0000", 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275768+0000", 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275777+0000", 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7200000000000001e-06 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275787+0000", 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6090000000000007e-06 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275795+0000", 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424019+0000", 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14822481500000001 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424030+0000", 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0197000000000001e-05 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41000 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414455452, 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14829276899999999, 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41000 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275805+0000", 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275809+0000", 2024-09-22T01:03:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275818+0000", 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3859999999999996e-06 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275827+0000", 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1030000000000008e-06 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424071+0000", 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14824412100000001 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424098+0000", 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6239000000000001e-05 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41001 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414423087, 2024-09-22T01:03:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14830634200000001, 2024-09-22T01:03:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41001 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275837+0000", 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275838+0000", 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275841+0000", 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9830000000000001e-06 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275850+0000", 2024-09-22T01:03:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2930000000000006e-06 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275858+0000", 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-06 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424130+0000", 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14827185700000001 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424143+0000", 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3852000000000001e-05 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41002 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414392768, 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148374487, 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41002 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275867+0000", 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275868+0000", 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275871+0000", 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0869999999999998e-06 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275880+0000", 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2769999999999994e-06 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275888+0000", 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3730000000000004e-06 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424213+0000", 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148324704 2024-09-22T01:03:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424242+0000", 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9403e-05 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41003 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414362876, 2024-09-22T01:03:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148432752, 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41003 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275897+0000", 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275898+0000", 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275901+0000", 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275910+0000", 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2869999999999993e-06 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275918+0000", 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424318+0000", 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14840081299999999 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424330+0000", 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1698999999999999e-05 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41004 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414333117, 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.148517279, 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41004 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275927+0000", 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275928+0000", 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275931+0000", 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.27e-06 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275947+0000", 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6133e-05 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275961+0000", 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3433e-05 2024-09-22T01:03:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424411+0000", 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14845089 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424444+0000", 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2932999999999998e-05 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:41005 3.d 3.2d40f02d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.414282652, 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14850160500000001, 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 41005 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.275978+0000", 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277650+0000", 2024-09-22T01:03:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001672261 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277659+0000", 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8359999999999998e-06 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277676+0000", 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7846000000000002e-05 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.277687+0000", 2024-09-22T01:03:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0519e-05 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424470+0000", 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.14678312900000001 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:32.424479+0000", 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:29.778 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:29.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:29.780 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:29.776+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:29.880 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:29.876+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:29.957 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:29.957 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:29.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:30.013 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:30.012+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:30.082 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:30.080+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:30.135 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:30.135 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:30.135 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:30.135 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:30.135 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:30.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:30.193 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:30.192+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:30.284 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:30.280+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.182460056, 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:30.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:03:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.182404381, 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:03:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:03:30.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:03:30.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:30.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.182352036, 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:30.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:03:30.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.182294125, 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.331 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:30.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:03:30.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.335 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.182235486, 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.336 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:03:30.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.182180254, 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:30.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:03:30.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.18212653000001, 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:03:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:03:30.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:03:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:30.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.182054795, 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:03:30.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.356 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.18199741399999, 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:30.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.361 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.18194714800001, 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.366 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.181868091, 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:03:30.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.181743842, 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.372 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:03:30.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:03:30.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.18166325999999, 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:03:30.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:03:30.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.181598072, 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:03:30.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.18082310299999, 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.18077331800001, 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:03:30.392 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:03:30.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:03:30.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:03:30.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.180692217, 2024-09-22T01:03:30.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:03:30.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.180622232, 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:03:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.180544868, 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:03:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:03:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.180484347, 2024-09-22T01:03:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:03:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:03:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:03:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:30.418 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:30.419 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:30.416+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:30.519 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:30.516+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:30.554 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:30.555 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:30.569 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:30.620 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:30.616+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:30.721 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:30.716+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:30.733 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:30.733 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:30.733 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:30.733 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:30.733 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:30.747 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:30.797 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:03:30.798 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:03:30.798 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-22T01:03:30.798 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:03:30.798 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:03:30.798 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-22T01:03:30.806 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:03:30.834 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:30.832+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:30.922 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:30.920+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:30.936 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:30.936 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:30.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:30.936 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:30.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.936 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:30.936 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:30.936 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 109.43503446699999, 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:30.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 109.435006624, 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.941 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:03:30.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:03:30.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:03:30.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 109.434974408, 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:03:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 109.43495267, 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:03:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:03:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:03:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 109.434933015, 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:03:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:03:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 109.434902001, 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:03:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:03:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479497789999996, 2024-09-22T01:03:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:03:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:03:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479458155000003, 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:03:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:03:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479370713999998, 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479330494999999, 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479294603, 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:03:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:03:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479246954999994, 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479212185999998, 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:03:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:03:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479178993000005, 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:03:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:03:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479134779000006, 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:03:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:03:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:03:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479092589999993, 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:03:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479058477999999, 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:03:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:03:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.479025551000007, 2024-09-22T01:03:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:03:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:03:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.478990783, 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:151264 3.1b 3:deb34d03:::rbd_object_map.117cf172dcd2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.94080203, 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.812141679, 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 151264 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993280+0000", 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8669999999999998e-06 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993295+0000", 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5235999999999999e-05 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993321+0000", 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5775e-05 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993335+0000", 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4306e-05 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993415+0000", 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9975999999999995e-05 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993426+0000", 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1063000000000001e-05 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978616+0000", 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98518988299999999 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978652+0000", 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5976000000000002e-05 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.803938+0000", 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.82528633799999995 2024-09-22T01:03:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804052+0000", 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113491 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804224+0000", 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172375 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805210+0000", 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000986385 2024-09-22T01:03:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805298+0000", 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7410000000000005e-05 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805343+0000", 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5451999999999998e-05 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805418+0000", 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5146000000000002e-05 2024-09-22T01:03:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.025 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:31.026 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "" 2024-09-22T01:03:31.027 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:31.027 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:31.027 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:31.024+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:31.127 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:31.124+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:31.200 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:31.200 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:31.214 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:31.258 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:31.256+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:31.329 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:31.324+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:31.340 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:31.340 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:31.354 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:31.442 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:31.440+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:31.497 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44661212599999, 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:03:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.446593941, 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.446576153, 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:03:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.446557595, 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:03:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.446539543, 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44652219400001, 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:03:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:03:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.446500241, 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:03:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44648270600001, 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:03:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44645892, 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:03:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:03:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.446441058, 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:03:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:03:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44641382099999, 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:03:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:03:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44639511599999, 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44637161899999, 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:03:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44634765000001, 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.446328435, 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:03:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:03:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.44532388499999, 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:03:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 156.442817932, 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:03:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 68.727359772, 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:03:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:03:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:03:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:140499 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.468683601999999, 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98072477000000002, 2024-09-22T01:03:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140499 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057286+0000", 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057297+0000", 2024-09-22T01:03:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026e-05 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057320+0000", 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3391999999999999e-05 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057331+0000", 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.075e-05 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057403+0000", 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057411+0000", 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2749999999999995e-06 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.037974+0000", 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98056295800000004 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.038008+0000", 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3844999999999999e-05 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:144964 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.744428042999999, 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95820099800000003, 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144964 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781542+0000", 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4899999999999999e-06 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781553+0000", 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1687e-05 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781583+0000", 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0173999999999999e-05 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781597+0000", 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3172e-05 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781681+0000", 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4810999999999998e-05 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781692+0000", 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0662000000000001e-05 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739463+0000", 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95777067000000005 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739603+0000", 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013200000000001 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739740+0000", 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001372 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:31.621 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:31.622 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:31.623 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:31.620+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:31.643 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:31.660 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:31.677 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:31.694 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:31.715 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:31.715 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:31.723 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:31.720+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:31.735 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:31.736 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:31.757 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:31.759 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:31.759 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:31.774 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:31.779 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:31.780 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:03:31.824 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:31.820+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:31.924 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:31.924 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:31.924 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:31.920+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:31.939 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:32.025 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:32.024+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:32.105 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:32.106 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:32.120 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:32.166 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:32.164+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:32.227 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:32.224+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:32.247 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:32.247 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:32.264 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:32.328 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:32.324+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:32.429 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:32.428+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:32.434 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:32.434 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:32.449 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:32.529 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:32.528+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:32.569 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:32.569 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:32.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:32.630 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:32.628+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:32.716 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:32.716 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:32.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:32.731 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:32.728+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:32.833 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:32.832+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:32.856 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:32.856 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:32.871 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:32.936 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:32.932+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:33.035 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:33.032+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:33.045 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:33.045 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:33.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:33.136 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:33.132+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:33.188 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:33.189 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:33.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:33.238 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:33.236+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:33.320 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:33.320 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:33.334 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:33.337 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:33.332+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:33.439 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:33.436+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:33.458 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:33.458 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:33.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:33.561 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:33.560+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:33.638 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:33.638 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:33.642 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:33.636+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:33.652 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:33.743 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:33.740+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:33.803 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:33.818 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:33.818 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:33.824 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:03:33.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:33.850 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:33.848+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:33.945 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:33.940+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:33.978 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:33.999 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:34.000 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:34.011 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:34.011 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:34.016 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:34.016 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:34.025 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:34.033 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:34.033 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:34.051 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:34.051 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:34.069 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:34.069 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:34.073 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:34.072+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:34.087 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:34.087 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:34.105 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:34.106 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:34.145 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:34.145 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:34.145 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-22T01:03:34.145 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:34.146 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:34.144+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:34.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:34.247 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:34.244+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:34.325 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-22T01:03:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-22T01:03:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-22T01:03:34.326 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:34.339 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:34.348 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:34.348+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:34.449 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:34.448+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.37450678899999, 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:03:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:03:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.374451114, 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.374398769, 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.374340858, 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.37428221899999, 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.374226987, 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.537 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.374173263, 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:34.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.542 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.374101528, 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.547 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.37404414700001, 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.373993881, 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:34.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:34.557 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.373914824, 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:34.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:03:34.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.562 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.373790575, 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:34.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:34.567 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.37370999300001, 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:34.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:03:34.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:03:34.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:03:34.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.373644805, 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:03:34.572 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:03:34.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:03:34.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:03:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.37286983600001, 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:03:34.577 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.37282005100001, 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:34.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:03:34.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:03:34.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:03:34.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.37273895, 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:03:34.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.372668965, 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.596 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.372591601, 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.601 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.37253108, 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:03:34.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:03:34.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:03:34.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:03:34.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.606 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:34.607 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:34.607 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:34.608 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:34.608 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:34.604+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:34.709 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:34.708+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:34.746 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:34.746 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:34.746 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:34.746 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:34.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:34.810 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:34.808+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:34.829 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-22T01:03:34.897 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:34.897 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:34.897 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:34.897 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:34.897 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:34.911 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:34.908+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:34.913 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:35.011 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:35.008+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:35.088 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.586271888, 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:03:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.586244045, 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:03:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.58621182900001, 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:03:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.58619009100001, 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:03:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:03:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:03:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.586170436, 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:03:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.586139422, 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:03:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630735211000001, 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:03:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630695575999994, 2024-09-22T01:03:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:03:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630608135000003, 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:03:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:03:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630567916000004, 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:03:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:03:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630532024000004, 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:03:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630484375999998, 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630449607000003, 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:03:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630416413999995, 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630372199999996, 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630330010999998, 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:03:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:03:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630295899000004, 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630262971999997, 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.630228204000005, 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:03:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:03:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:03:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:151264 3.1b 3:deb34d03:::rbd_object_map.117cf172dcd2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.092039451000002, 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.812141679, 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 151264 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993280+0000", 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8669999999999998e-06 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993295+0000", 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5235999999999999e-05 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993321+0000", 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5775e-05 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993335+0000", 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4306e-05 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993415+0000", 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9975999999999995e-05 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993426+0000", 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1063000000000001e-05 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978616+0000", 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98518988299999999 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978652+0000", 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5976000000000002e-05 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.803938+0000", 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.82528633799999995 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804052+0000", 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113491 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804224+0000", 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172375 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805210+0000", 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000986385 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805298+0000", 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7410000000000005e-05 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805343+0000", 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5451999999999998e-05 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805418+0000", 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5146000000000002e-05 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.178 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:35.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:35.180 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:35.176+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:35.282 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:35.276+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:35.354 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:35.354 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:35.354 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:35.354 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:35.369 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:35.381 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:35.380+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:35.482 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:35.480+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:35.498 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:35.498 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:35.498 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:35.498 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:35.498 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:35.512 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:35.583 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:35.580+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:35.684 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:35.680+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:35.687 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:35.687 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60395126700001, 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:03:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.603933082, 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60391529399999, 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:03:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.603896736, 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:03:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60387868399999, 2024-09-22T01:03:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.603861335, 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:03:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:03:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60383938199999, 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:03:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60382184700001, 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:03:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60379806099999, 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:03:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.603780199, 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:03:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60375296199999, 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:03:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:03:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:03:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:03:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:03:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60373425700001, 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:03:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60371076000001, 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:03:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:03:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.603686791, 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60366757599999, 2024-09-22T01:03:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:03:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:03:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60266302599999, 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:03:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 160.60015707299999, 2024-09-22T01:03:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:03:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:03:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.884698912999994, 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:03:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:140499 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.626022743, 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98072477000000002, 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140499 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057286+0000", 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057297+0000", 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026e-05 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057320+0000", 2024-09-22T01:03:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3391999999999999e-05 2024-09-22T01:03:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057331+0000", 2024-09-22T01:03:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.075e-05 2024-09-22T01:03:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057403+0000", 2024-09-22T01:03:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:03:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057411+0000", 2024-09-22T01:03:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2749999999999995e-06 2024-09-22T01:03:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.037974+0000", 2024-09-22T01:03:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98056295800000004 2024-09-22T01:03:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.038008+0000", 2024-09-22T01:03:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3844999999999999e-05 2024-09-22T01:03:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:144964 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:03:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.901767184000001, 2024-09-22T01:03:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95820099800000003, 2024-09-22T01:03:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144964 2024-09-22T01:03:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781542+0000", 2024-09-22T01:03:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4899999999999999e-06 2024-09-22T01:03:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781553+0000", 2024-09-22T01:03:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1687e-05 2024-09-22T01:03:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781583+0000", 2024-09-22T01:03:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0173999999999999e-05 2024-09-22T01:03:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781597+0000", 2024-09-22T01:03:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3172e-05 2024-09-22T01:03:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781681+0000", 2024-09-22T01:03:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4810999999999998e-05 2024-09-22T01:03:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781692+0000", 2024-09-22T01:03:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0662000000000001e-05 2024-09-22T01:03:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739463+0000", 2024-09-22T01:03:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95777067000000005 2024-09-22T01:03:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739603+0000", 2024-09-22T01:03:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013200000000001 2024-09-22T01:03:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739740+0000", 2024-09-22T01:03:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001372 2024-09-22T01:03:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:35.800 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:35.800 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:35.796+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:35.899 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:35.896+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:35.988 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:35.988 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:36.000 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:35.996+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:36.004 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:36.028 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:03:36.028 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:03:36.029 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:03:36.103 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:36.100+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:36.124 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:36.154 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:36.203 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:36.200+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:36.278 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:36.278 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:36.296 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:36.297 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:36.304 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:36.300+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:36.314 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:36.315 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:36.317 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:36.317 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:36.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:36.333 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:36.334 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:36.354 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:36.354 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:03:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:03:36.375 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:36.392 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-22T01:03:35.839475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151743613,"num_objects":522,"num_object_clones":99,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522,"num_whiteouts":344,"num_read":201402,"num_read_kb":758875,"num_write":22453,"num_write_kb":413444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19501,"ondisk_log_size":19501,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409544,"kb_used_data":50628,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754565176,"statfs":{"total":773094113280,"available":772674740224,"internally_reserved":0,"allocated":51843072,"data_stored":46753653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-106589211,"num_objects":-115,"num_object_clones":-39,"num_object_copies":-230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-115,"num_whiteouts":-33,"num_read":1215,"num_read_kb":1037,"num_write":167,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.045147"},"pg_stats":[{"pgid":"3.1f","version":"168'525","reported_seq":7103,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.843215+0000","last_change":"2024-09-22T01:03:34.843546+0000","last_active":"2024-09-22T01:03:34.843215+0000","last_peered":"2024-09-22T01:03:34.843215+0000","last_clean":"2024-09-22T01:03:34.843215+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:03:34.843215+0000","last_undegraded":"2024-09-22T01:03:34.843215+0000","last_fullsized":"2024-09-22T01:03:34.843215+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:11:51.486044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074412130000000003,"stat_sum":{"num_bytes":5742608,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":6535,"num_read_kb":45963,"num_write":553,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.1e","version":"168'762","reported_seq":3624,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:35.074892+0000","last_change":"2024-09-22T01:03:34.835267+0000","last_active":"2024-09-22T01:03:35.074892+0000","last_peered":"2024-09-22T01:03:35.074892+0000","last_clean":"2024-09-22T01:03:35.074892+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:03:35.074892+0000","last_undegraded":"2024-09-22T01:03:35.074892+0000","last_fullsized":"2024-09-22T01:03:35.074892+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:33:56.629548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022543569999999998,"stat_sum":{"num_bytes":805176,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":10,"num_read":6361,"num_read_kb":12994,"num_write":962,"num_write_kb":7199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.1d","version":"160'465","reported_seq":5085,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.823919+0000","last_change":"2024-09-22T01:03:30.824308+0000","last_active":"2024-09-22T01:03:30.823919+0000","last_peered":"2024-09-22T01:03:30.823919+0000","last_clean":"2024-09-22T01:03:30.823919+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:03:30.823919+0000","last_undegraded":"2024-09-22T01:03:30.823919+0000","last_fullsized":"2024-09-22T01:03:30.823919+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:09:14.537628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012246178999999999,"stat_sum":{"num_bytes":2293790,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":4787,"num_read_kb":32328,"num_write":478,"num_write_kb":18066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"7"}]},{"pgid":"3.1c","version":"165'1102","reported_seq":10535,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.920351+0000","last_change":"2024-09-22T01:03:30.823843+0000","last_active":"2024-09-22T01:03:30.920351+0000","last_peered":"2024-09-22T01:03:30.920351+0000","last_clean":"2024-09-22T01:03:30.920351+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:03:30.920351+0000","last_undegraded":"2024-09-22T01:03:30.920351+0000","last_fullsized":"2024-09-22T01:03:30.920351+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1102,"log_dups_size":0,"ondisk_log_size":1102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:02:15.775734+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013822431,"stat_sum":{"num_bytes":3143824,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":11436,"num_read_kb":51667,"num_write":1581,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"7"}]},{"pgid":"3.1b","version":"168'1081","reported_seq":3035,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.842697+0000","last_change":"2024-09-22T01:03:34.843202+0000","last_active":"2024-09-22T01:03:34.842697+0000","last_peered":"2024-09-22T01:03:34.842697+0000","last_clean":"2024-09-22T01:03:34.842697+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:03:34.842697+0000","last_undegraded":"2024-09-22T01:03:34.842697+0000","last_fullsized":"2024-09-22T01:03:34.842697+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:26:49.885604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070593410000000002,"stat_sum":{"num_bytes":2313015,"num_objects":20,"num_object_clones":8,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":11,"num_read":5211,"num_read_kb":15422,"num_write":1521,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.1a","version":"166'478","reported_seq":12253,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:35.074052+0000","last_change":"2024-09-22T01:03:34.838921+0000","last_active":"2024-09-22T01:03:35.074052+0000","last_peered":"2024-09-22T01:03:35.074052+0000","last_clean":"2024-09-22T01:03:35.074052+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:03:35.074052+0000","last_undegraded":"2024-09-22T01:03:35.074052+0000","last_fullsized":"2024-09-22T01:03:35.074052+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:48:16.867650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023023800000000001,"stat_sum":{"num_bytes":3498003,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":10,"num_read":12133,"num_read_kb":25973,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.19","version":"159'392","reported_seq":9988,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.828326+0000","last_change":"2024-09-22T01:03:34.829052+0000","last_active":"2024-09-22T01:03:34.828326+0000","last_peered":"2024-09-22T01:03:34.828326+0000","last_clean":"2024-09-22T01:03:34.828326+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:03:34.828326+0000","last_undegraded":"2024-09-22T01:03:34.828326+0000","last_fullsized":"2024-09-22T01:03:34.828326+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:09.312294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010304960000000001,"stat_sum":{"num_bytes":6073841,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":9135,"num_read_kb":31303,"num_write":453,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.18","version":"169'767","reported_seq":6473,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:35.111816+0000","last_change":"2024-09-22T01:03:34.834496+0000","last_active":"2024-09-22T01:03:35.111816+0000","last_peered":"2024-09-22T01:03:35.111816+0000","last_clean":"2024-09-22T01:03:35.111816+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:03:35.111816+0000","last_undegraded":"2024-09-22T01:03:35.111816+0000","last_fullsized":"2024-09-22T01:03:35.111816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:29:12.485652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020930499999999999,"stat_sum":{"num_bytes":13100138,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":8511,"num_read_kb":42633,"num_write":839,"num_write_kb":16850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.17","version":"160'552","reported_seq":4278,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.809388+0000","last_change":"2024-09-22T01:03:30.810098+0000","last_active":"2024-09-22T01:03:30.809388+0000","last_peered":"2024-09-22T01:03:30.809388+0000","last_clean":"2024-09-22T01:03:30.809388+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:03:30.809388+0000","last_undegraded":"2024-09-22T01:03:30.809388+0000","last_fullsized":"2024-09-22T01:03:30.809388+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:35:16.147488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032222969999999998,"stat_sum":{"num_bytes":5605161,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":4907,"num_read_kb":18393,"num_write":625,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"e","length":"7"}]},{"pgid":"3.16","version":"165'359","reported_seq":2149,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.917922+0000","last_change":"2024-09-22T01:03:30.821371+0000","last_active":"2024-09-22T01:03:30.917922+0000","last_peered":"2024-09-22T01:03:30.917922+0000","last_clean":"2024-09-22T01:03:30.917922+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:03:30.917922+0000","last_undegraded":"2024-09-22T01:03:30.917922+0000","last_fullsized":"2024-09-22T01:03:30.917922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:32:01.164436+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011578079,"stat_sum":{"num_bytes":3121291,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":2587,"num_read_kb":34288,"num_write":389,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"7"}]},{"pgid":"3.15","version":"168'661","reported_seq":6694,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:35.032808+0000","last_change":"2024-09-22T01:03:34.833319+0000","last_active":"2024-09-22T01:03:35.032808+0000","last_peered":"2024-09-22T01:03:35.032808+0000","last_clean":"2024-09-22T01:03:35.032808+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:03:35.032808+0000","last_undegraded":"2024-09-22T01:03:35.032808+0000","last_fullsized":"2024-09-22T01:03:35.032808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:33:44.371472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027818069999999999,"stat_sum":{"num_bytes":3883805,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":11,"num_read":8728,"num_read_kb":26989,"num_write":681,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.14","version":"167'550","reported_seq":13877,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.836914+0000","last_change":"2024-09-22T01:03:34.837962+0000","last_active":"2024-09-22T01:03:34.836914+0000","last_peered":"2024-09-22T01:03:34.836914+0000","last_clean":"2024-09-22T01:03:34.836914+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:03:34.836914+0000","last_undegraded":"2024-09-22T01:03:34.836914+0000","last_fullsized":"2024-09-22T01:03:34.836914+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:40:55.985511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001699683,"stat_sum":{"num_bytes":6828712,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":13906,"num_read_kb":40986,"num_write":565,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":308,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.807255+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:03:30.807255+0000","last_peered":"2024-09-22T01:03:30.807255+0000","last_clean":"2024-09-22T01:03:30.807255+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:03:30.807255+0000","last_undegraded":"2024-09-22T01:03:30.807255+0000","last_fullsized":"2024-09-22T01:03:30.807255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"169'597","reported_seq":3219,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.843705+0000","last_change":"2024-09-22T01:03:34.843705+0000","last_active":"2024-09-22T01:03:34.843705+0000","last_peered":"2024-09-22T01:03:34.843705+0000","last_clean":"2024-09-22T01:03:34.843705+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:03:34.843705+0000","last_undegraded":"2024-09-22T01:03:34.843705+0000","last_fullsized":"2024-09-22T01:03:34.843705+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:34:03.041566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075673980000000004,"stat_sum":{"num_bytes":145281,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":557,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":130,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.828596+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:03:34.828596+0000","last_peered":"2024-09-22T01:03:34.828596+0000","last_clean":"2024-09-22T01:03:34.828596+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:03:34.828596+0000","last_undegraded":"2024-09-22T01:03:34.828596+0000","last_fullsized":"2024-09-22T01:03:34.828596+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"166'545","reported_seq":3537,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.835349+0000","last_change":"2024-09-22T01:03:34.835824+0000","last_active":"2024-09-22T01:03:34.835349+0000","last_peered":"2024-09-22T01:03:34.835349+0000","last_clean":"2024-09-22T01:03:34.835349+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:03:34.835349+0000","last_undegraded":"2024-09-22T01:03:34.835349+0000","last_fullsized":"2024-09-22T01:03:34.835349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:49:46.769634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002754917,"stat_sum":{"num_bytes":8351091,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":12,"num_read":4467,"num_read_kb":12697,"num_write":676,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":316,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.830891+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:03:34.830891+0000","last_peered":"2024-09-22T01:03:34.830891+0000","last_clean":"2024-09-22T01:03:34.830891+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:03:34.830891+0000","last_undegraded":"2024-09-22T01:03:34.830891+0000","last_fullsized":"2024-09-22T01:03:34.830891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"167'431","reported_seq":4411,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.829152+0000","last_change":"2024-09-22T01:03:34.829657+0000","last_active":"2024-09-22T01:03:34.829152+0000","last_peered":"2024-09-22T01:03:34.829152+0000","last_clean":"2024-09-22T01:03:34.829152+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:03:34.829152+0000","last_undegraded":"2024-09-22T01:03:34.829152+0000","last_fullsized":"2024-09-22T01:03:34.829152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:48:57.043427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001270223,"stat_sum":{"num_bytes":2782177,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":15,"num_read":4002,"num_read_kb":26479,"num_write":409,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.828172+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:03:34.828172+0000","last_peered":"2024-09-22T01:03:34.828172+0000","last_clean":"2024-09-22T01:03:34.828172+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:03:34.828172+0000","last_undegraded":"2024-09-22T01:03:34.828172+0000","last_fullsized":"2024-09-22T01:03:34.828172+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"165'631","reported_seq":4367,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.976748+0000","last_change":"2024-09-22T01:03:30.825152+0000","last_active":"2024-09-22T01:03:30.976748+0000","last_peered":"2024-09-22T01:03:30.976748+0000","last_clean":"2024-09-22T01:03:30.976748+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:03:30.976748+0000","last_undegraded":"2024-09-22T01:03:30.976748+0000","last_fullsized":"2024-09-22T01:03:30.976748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:54:22.971770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012229477000000001,"stat_sum":{"num_bytes":1843268,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":6871,"num_read_kb":12630,"num_write":557,"num_write_kb":5646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"7"}]},{"pgid":"3.3","version":"169'683","reported_seq":2918,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.976382+0000","last_change":"2024-09-22T01:03:34.838951+0000","last_active":"2024-09-22T01:03:34.976382+0000","last_peered":"2024-09-22T01:03:34.976382+0000","last_clean":"2024-09-22T01:03:34.976382+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:03:34.976382+0000","last_undegraded":"2024-09-22T01:03:34.976382+0000","last_fullsized":"2024-09-22T01:03:34.976382+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:43.091426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026489539999999998,"stat_sum":{"num_bytes":9818790,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":3336,"num_read_kb":17951,"num_write":710,"num_write_kb":15202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"2.2","version":"21'2","reported_seq":281,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.814753+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:03:30.814753+0000","last_peered":"2024-09-22T01:03:30.814753+0000","last_clean":"2024-09-22T01:03:30.814753+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:03:30.814753+0000","last_undegraded":"2024-09-22T01:03:30.814753+0000","last_fullsized":"2024-09-22T01:03:30.814753+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"168'1646","reported_seq":3845,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.828419+0000","last_change":"2024-09-22T01:03:34.829053+0000","last_active":"2024-09-22T01:03:34.828419+0000","last_peered":"2024-09-22T01:03:34.828419+0000","last_clean":"2024-09-22T01:03:34.828419+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:03:34.828419+0000","last_undegraded":"2024-09-22T01:03:34.828419+0000","last_fullsized":"2024-09-22T01:03:34.828419+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1646,"log_dups_size":0,"ondisk_log_size":1646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:31.351188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010219890000000001,"stat_sum":{"num_bytes":3749078,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":9103,"num_read_kb":25480,"num_write":2700,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":316,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.827567+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:03:34.827567+0000","last_peered":"2024-09-22T01:03:34.827567+0000","last_clean":"2024-09-22T01:03:34.827567+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:03:34.827567+0000","last_undegraded":"2024-09-22T01:03:34.827567+0000","last_fullsized":"2024-09-22T01:03:34.827567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"168'381","reported_seq":3067,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.842609+0000","last_change":"2024-09-22T01:03:34.843141+0000","last_active":"2024-09-22T01:03:34.842609+0000","last_peered":"2024-09-22T01:03:34.842609+0000","last_clean":"2024-09-22T01:03:34.842609+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:03:34.842609+0000","last_undegraded":"2024-09-22T01:03:34.842609+0000","last_fullsized":"2024-09-22T01:03:34.842609+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:12.636884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00724627,"stat_sum":{"num_bytes":2006464,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":11,"num_read":2222,"num_read_kb":18605,"num_write":394,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":331,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.829855+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:03:34.829855+0000","last_peered":"2024-09-22T01:03:34.829855+0000","last_clean":"2024-09-22T01:03:34.829855+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:03:34.829855+0000","last_undegraded":"2024-09-22T01:03:34.829855+0000","last_fullsized":"2024-09-22T01:03:34.829855+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"166'376","reported_seq":2292,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.833425+0000","last_change":"2024-09-22T01:03:34.833845+0000","last_active":"2024-09-22T01:03:34.833425+0000","last_peered":"2024-09-22T01:03:34.833425+0000","last_clean":"2024-09-22T01:03:34.833425+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:03:34.833425+0000","last_undegraded":"2024-09-22T01:03:34.833425+0000","last_fullsized":"2024-09-22T01:03:34.833425+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:54:12.788100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001481911,"stat_sum":{"num_bytes":7869969,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":1659,"num_read_kb":15604,"num_write":373,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"2.3","version":"19'1","reported_seq":262,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.809539+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:03:30.809539+0000","last_peered":"2024-09-22T01:03:30.809539+0000","last_clean":"2024-09-22T01:03:30.809539+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:03:30.809539+0000","last_undegraded":"2024-09-22T01:03:30.809539+0000","last_fullsized":"2024-09-22T01:03:30.809539+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":467,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.829953+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:03:34.829953+0000","last_peered":"2024-09-22T01:03:34.829953+0000","last_clean":"2024-09-22T01:03:34.829953+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:03:34.829953+0000","last_undegraded":"2024-09-22T01:03:34.829953+0000","last_fullsized":"2024-09-22T01:03:34.829953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"167'482","reported_seq":10694,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.833530+0000","last_change":"2024-09-22T01:03:34.834023+0000","last_active":"2024-09-22T01:03:34.833530+0000","last_peered":"2024-09-22T01:03:34.833530+0000","last_clean":"2024-09-22T01:03:34.833530+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:03:34.833530+0000","last_undegraded":"2024-09-22T01:03:34.833530+0000","last_fullsized":"2024-09-22T01:03:34.833530+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:23:32.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016558110000000001,"stat_sum":{"num_bytes":1824953,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":15,"num_read":10962,"num_read_kb":27619,"num_write":452,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.9","version":"167'597","reported_seq":2489,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.836736+0000","last_change":"2024-09-22T01:03:34.837759+0000","last_active":"2024-09-22T01:03:34.836736+0000","last_peered":"2024-09-22T01:03:34.836736+0000","last_clean":"2024-09-22T01:03:34.836736+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:03:34.836736+0000","last_undegraded":"2024-09-22T01:03:34.836736+0000","last_fullsized":"2024-09-22T01:03:34.836736+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:45.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015176110000000001,"stat_sum":{"num_bytes":5681823,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":12,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.a","version":"165'775","reported_seq":16311,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.809286+0000","last_change":"2024-09-22T01:03:30.809286+0000","last_active":"2024-09-22T01:03:30.809286+0000","last_peered":"2024-09-22T01:03:30.809286+0000","last_clean":"2024-09-22T01:03:30.809286+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:03:30.809286+0000","last_undegraded":"2024-09-22T01:03:30.809286+0000","last_fullsized":"2024-09-22T01:03:30.809286+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:33:17.231820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026056310000000002,"stat_sum":{"num_bytes":5643671,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":16596,"num_read_kb":40544,"num_write":798,"num_write_kb":13037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"e","length":"7"}]},{"pgid":"3.b","version":"169'672","reported_seq":2796,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:35.112716+0000","last_change":"2024-09-22T01:03:34.836350+0000","last_active":"2024-09-22T01:03:35.112716+0000","last_peered":"2024-09-22T01:03:35.112716+0000","last_clean":"2024-09-22T01:03:35.112716+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:03:35.112716+0000","last_undegraded":"2024-09-22T01:03:35.112716+0000","last_fullsized":"2024-09-22T01:03:35.112716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:37:27.209102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023667810000000001,"stat_sum":{"num_bytes":2612796,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":12,"num_read":4349,"num_read_kb":19882,"num_write":800,"num_write_kb":10860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.c","version":"160'408","reported_seq":1458,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.821543+0000","last_change":"2024-09-22T01:03:30.823872+0000","last_active":"2024-09-22T01:03:30.821543+0000","last_peered":"2024-09-22T01:03:30.821543+0000","last_clean":"2024-09-22T01:03:30.821543+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:03:30.821543+0000","last_undegraded":"2024-09-22T01:03:30.821543+0000","last_fullsized":"2024-09-22T01:03:30.821543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:59:48.867025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013258479,"stat_sum":{"num_bytes":11953098,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":2805,"num_read_kb":21867,"num_write":398,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"7"}]},{"pgid":"3.d","version":"166'751","reported_seq":5925,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.834777+0000","last_change":"2024-09-22T01:03:34.836244+0000","last_active":"2024-09-22T01:03:34.834777+0000","last_peered":"2024-09-22T01:03:34.834777+0000","last_clean":"2024-09-22T01:03:34.834777+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:03:34.834777+0000","last_undegraded":"2024-09-22T01:03:34.834777+0000","last_fullsized":"2024-09-22T01:03:34.834777+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:21.888584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051086689999999997,"stat_sum":{"num_bytes":3760128,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":12,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.e","version":"169'479","reported_seq":5116,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.970549+0000","last_change":"2024-09-22T01:03:34.838195+0000","last_active":"2024-09-22T01:03:34.970549+0000","last_peered":"2024-09-22T01:03:34.970549+0000","last_clean":"2024-09-22T01:03:34.970549+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:03:34.970549+0000","last_undegraded":"2024-09-22T01:03:34.970549+0000","last_fullsized":"2024-09-22T01:03:34.970549+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:09:14.944041+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0076780370000000004,"stat_sum":{"num_bytes":5943373,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":10,"num_read":3857,"num_read_kb":26287,"num_write":515,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.f","version":"168'373","reported_seq":5034,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:35.046998+0000","last_change":"2024-09-22T01:03:34.834594+0000","last_active":"2024-09-22T01:03:35.046998+0000","last_peered":"2024-09-22T01:03:35.046998+0000","last_clean":"2024-09-22T01:03:35.046998+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:03:35.046998+0000","last_undegraded":"2024-09-22T01:03:35.046998+0000","last_fullsized":"2024-09-22T01:03:35.046998+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:29.646536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003862627,"stat_sum":{"num_bytes":3407872,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":5072,"num_read_kb":10257,"num_write":295,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.10","version":"165'451","reported_seq":3279,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T01:03:30.821985+0000","last_change":"2024-09-22T01:03:30.821985+0000","last_active":"2024-09-22T01:03:30.821985+0000","last_peered":"2024-09-22T01:03:30.821985+0000","last_clean":"2024-09-22T01:03:30.821985+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:03:30.821985+0000","last_undegraded":"2024-09-22T01:03:30.821985+0000","last_fullsized":"2024-09-22T01:03:30.821985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:10:43.939443+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015167154,"stat_sum":{"num_bytes":7404991,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":3238,"num_read_kb":14715,"num_write":391,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"e","length":"7"}]},{"pgid":"3.11","version":"168'656","reported_seq":2943,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.831728+0000","last_change":"2024-09-22T01:03:34.831728+0000","last_active":"2024-09-22T01:03:34.831728+0000","last_peered":"2024-09-22T01:03:34.831728+0000","last_clean":"2024-09-22T01:03:34.831728+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:03:34.831728+0000","last_undegraded":"2024-09-22T01:03:34.831728+0000","last_fullsized":"2024-09-22T01:03:34.831728+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:08.682473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001436524,"stat_sum":{"num_bytes":9541794,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":4512,"num_read_kb":11729,"num_write":783,"num_write_kb":5789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.12","version":"166'441","reported_seq":3173,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.843633+0000","last_change":"2024-09-22T01:03:34.844106+0000","last_active":"2024-09-22T01:03:34.843633+0000","last_peered":"2024-09-22T01:03:34.843633+0000","last_clean":"2024-09-22T01:03:34.843633+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:03:34.843633+0000","last_undegraded":"2024-09-22T01:03:34.843633+0000","last_fullsized":"2024-09-22T01:03:34.843633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:37:53.567241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079036969999999995,"stat_sum":{"num_bytes":263981,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"7"}]},{"pgid":"3.13","version":"166'312","reported_seq":6805,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-22T01:03:34.834250+0000","last_change":"2024-09-22T01:03:34.834609+0000","last_active":"2024-09-22T01:03:34.834250+0000","last_peered":"2024-09-22T01:03:34.834250+0000","last_clean":"2024-09-22T01:03:34.834250+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:03:34.834250+0000","last_undegraded":"2024-09-22T01:03:34.834250+0000","last_fullsized":"2024-09-22T01:03:34.834250+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:13:11.101308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617229999999999,"stat_sum":{"num_bytes":139264,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":151153226,"num_objects":518,"num_object_clones":99,"num_object_copies":1036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":518,"num_whiteouts":344,"num_read":201286,"num_read_kb":758775,"num_write":22292,"num_write_kb":411026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":29286400,"data_stored":32568575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19381,"ondisk_log_size":19381,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34644,"kb_used_data":3452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337196,"statfs":{"total":96636764160,"available":96601288704,"internally_reserved":0,"allocated":3534848,"data_stored":2458882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299418,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37140,"kb_used_data":9020,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94334700,"statfs":{"total":96636764160,"available":96598732800,"internally_reserved":0,"allocated":9236480,"data_stored":8329565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64236,"kb_used_data":6672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94307604,"statfs":{"total":96636764160,"available":96570986496,"internally_reserved":0,"allocated":6832128,"data_stored":6507674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62632,"kb_used_data":9236,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309208,"statfs":{"total":96636764160,"available":96572628992,"internally_reserved":0,"allocated":9457664,"data_stored":9088589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52852,"kb_used_data":7768,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318988,"statfs":{"total":96636764160,"available":96582643712,"internally_reserved":0,"allocated":7954432,"data_stored":7806279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51888,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319952,"statfs":{"total":96636764160,"available":96583630848,"internally_reserved":0,"allocated":2711552,"data_stored":1763335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53156,"kb_used_data":3908,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318684,"statfs":{"total":96636764160,"available":96582332416,"internally_reserved":0,"allocated":4001792,"data_stored":3212479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52996,"kb_used_data":7924,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318844,"statfs":{"total":96636764160,"available":96582496256,"internally_reserved":0,"allocated":8114176,"data_stored":7586850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5492736,"data_stored":6018627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1970176,"data_stored":2234605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":684032,"data_stored":785480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5926912,"data_stored":6828424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6619136,"data_stored":7299524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2826240,"data_stored":2956137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1556480,"data_stored":1506327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4939451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:03:36.396 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:36.396 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:36.404 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:36.400+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:36.417 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:36.417 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:36.450 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:36.451 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:36.465 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:36.506 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:36.504+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:36.606 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:36.604+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:36.625 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:36.626 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:36.640 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:36.708 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:36.704+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:36.808 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:36.808 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:36.808 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:36.804+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:36.822 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:36.861 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:03:36.909 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:36.908+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:36.966 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:36.966 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:36.982 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:37.025 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:37.024+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:37.111 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:37.108+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:37.147 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:37.147 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:37.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:37.212 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:37.208+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:37.285 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:37.285 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:37.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:37.313 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:37.308+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:37.414 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:37.412+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:37.424 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:37.424 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:37.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:37.525 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:37.524+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:37.600 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:37.601 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:37.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:37.616 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:37.612+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:37.717 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:37.716+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:37.742 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-22T01:03:37.788 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:37.788 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:37.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:37.845 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:37.844+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:37.918 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:37.916+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:37.968 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:37.968 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:37.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:38.025 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:38.024+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:38.121 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:38.116+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:38.149 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:38.149 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:38.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:38.221 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:38.220+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:38.282 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:38.282 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:38.296 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:38.322 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:38.320+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:38.423 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:38.420+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:38.436 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:03:38.462 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:38.463 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:38.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:38.524 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-22T01:03:38.520+0000 7f092d60b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26642) UID: 0 2024-09-22T01:03:38.594 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:38.600 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:38.601 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:38.611 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:38.611 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:38.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:38.625 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:38.620+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:38.628 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:38.628 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:38.646 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:38.667 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:38.688 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:38.688 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:38.706 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:38.706 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:38.723 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:38.723 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:38.726 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:38.724+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51655 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1703936~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.651656219, 2024-09-22T01:03:38.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.098446021, 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51655 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129253+0000", 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129254+0000", 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:38.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129258+0000", 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7400000000000002e-06 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129275+0000", 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7646999999999999e-05 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129283+0000", 2024-09-22T01:03:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136305+0000", 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070224379999999998 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136311+0000", 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227512+0000", 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091201136 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227582+0000", 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0593999999999995e-05 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227699+0000", 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011701600000000001 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.789 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51656 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1732608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.651600544, 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0987931959999999, 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51656 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129309+0000", 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129310+0000", 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129313+0000", 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129333+0000", 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0380000000000001e-05 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129344+0000", 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0692e-05 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136358+0000", 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070136019999999999 2024-09-22T01:03:38.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136371+0000", 2024-09-22T01:03:38.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3456e-05 2024-09-22T01:03:38.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227738+0000", 2024-09-22T01:03:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091366566 2024-09-22T01:03:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.227837+0000", 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9202000000000003e-05 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228102+0000", 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000265235 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51657 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1761280~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.651548199, 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0989614839999999, 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51657 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129361+0000", 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129362+0000", 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:03:38.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129365+0000", 2024-09-22T01:03:38.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000001e-06 2024-09-22T01:03:38.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129381+0000", 2024-09-22T01:03:38.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5682e-05 2024-09-22T01:03:38.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129392+0000", 2024-09-22T01:03:38.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-22T01:03:38.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136410+0000", 2024-09-22T01:03:38.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070179259999999999 2024-09-22T01:03:38.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136415+0000", 2024-09-22T01:03:38.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4639999999999997e-06 2024-09-22T01:03:38.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228138+0000", 2024-09-22T01:03:38.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917231860000001 2024-09-22T01:03:38.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228207+0000", 2024-09-22T01:03:38.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8782000000000007e-05 2024-09-22T01:03:38.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228323+0000", 2024-09-22T01:03:38.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115537 2024-09-22T01:03:38.805 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.805 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.805 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.806 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51658 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1789952~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.806 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:38.806 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.65149028800001, 2024-09-22T01:03:38.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0993647360000001, 2024-09-22T01:03:38.807 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.807 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.807 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.807 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51658 2024-09-22T01:03:38.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.808 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:38.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129419+0000", 2024-09-22T01:03:38.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129420+0000", 2024-09-22T01:03:38.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7700000000000004e-07 2024-09-22T01:03:38.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129423+0000", 2024-09-22T01:03:38.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-22T01:03:38.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129443+0000", 2024-09-22T01:03:38.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9289000000000001e-05 2024-09-22T01:03:38.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129450+0000", 2024-09-22T01:03:38.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:38.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136464+0000", 2024-09-22T01:03:38.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070146050000000001 2024-09-22T01:03:38.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136478+0000", 2024-09-22T01:03:38.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3229e-05 2024-09-22T01:03:38.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228373+0000", 2024-09-22T01:03:38.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0918958489999999 2024-09-22T01:03:38.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228486+0000", 2024-09-22T01:03:38.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112125 2024-09-22T01:03:38.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228784+0000", 2024-09-22T01:03:38.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029840900000000002 2024-09-22T01:03:38.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.817 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.818 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51659 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1818624~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.818 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:38.818 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.651431649, 2024-09-22T01:03:38.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0995779539999999, 2024-09-22T01:03:38.818 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.819 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.819 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51659 2024-09-22T01:03:38.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129478+0000", 2024-09-22T01:03:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129479+0000", 2024-09-22T01:03:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:03:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129482+0000", 2024-09-22T01:03:38.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.303e-06 2024-09-22T01:03:38.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129499+0000", 2024-09-22T01:03:38.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7433000000000001e-05 2024-09-22T01:03:38.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129506+0000", 2024-09-22T01:03:38.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:03:38.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136518+0000", 2024-09-22T01:03:38.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070115910000000002 2024-09-22T01:03:38.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136523+0000", 2024-09-22T01:03:38.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3340000000000001e-06 2024-09-22T01:03:38.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228831+0000", 2024-09-22T01:03:38.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0923079330000001 2024-09-22T01:03:38.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.228911+0000", 2024-09-22T01:03:38.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9359999999999999e-05 2024-09-22T01:03:38.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229056+0000", 2024-09-22T01:03:38.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014516400000000001 2024-09-22T01:03:38.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.829 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.829 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51660 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1847296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.830 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:38.830 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.65137641699999, 2024-09-22T01:03:38.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.099903702, 2024-09-22T01:03:38.830 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.830 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.831 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.831 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51660 2024-09-22T01:03:38.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.831 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:38.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129533+0000", 2024-09-22T01:03:38.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129534+0000", 2024-09-22T01:03:38.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:38.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129537+0000", 2024-09-22T01:03:38.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:03:38.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129553+0000", 2024-09-22T01:03:38.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6045999999999998e-05 2024-09-22T01:03:38.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129561+0000", 2024-09-22T01:03:38.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2840000000000006e-06 2024-09-22T01:03:38.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136573+0000", 2024-09-22T01:03:38.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070112569999999999 2024-09-22T01:03:38.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136586+0000", 2024-09-22T01:03:38.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3467e-05 2024-09-22T01:03:38.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229113+0000", 2024-09-22T01:03:38.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0925267299999999 2024-09-22T01:03:38.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229253+0000", 2024-09-22T01:03:38.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140254 2024-09-22T01:03:38.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229437+0000", 2024-09-22T01:03:38.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000183635 2024-09-22T01:03:38.840 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.841 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.841 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51661 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1875968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.841 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:38.842 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.651322693, 2024-09-22T01:03:38.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.100130563, 2024-09-22T01:03:38.842 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.842 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.842 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.843 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.843 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.843 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51661 2024-09-22T01:03:38.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.843 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:38.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129587+0000", 2024-09-22T01:03:38.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129588+0000", 2024-09-22T01:03:38.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7400000000000003e-07 2024-09-22T01:03:38.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129591+0000", 2024-09-22T01:03:38.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5360000000000001e-06 2024-09-22T01:03:38.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129605+0000", 2024-09-22T01:03:38.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3903000000000001e-05 2024-09-22T01:03:38.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129616+0000", 2024-09-22T01:03:38.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.059e-05 2024-09-22T01:03:38.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136627+0000", 2024-09-22T01:03:38.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070111649999999998 2024-09-22T01:03:38.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136632+0000", 2024-09-22T01:03:38.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4559999999999999e-06 2024-09-22T01:03:38.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229474+0000", 2024-09-22T01:03:38.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.092841717 2024-09-22T01:03:38.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229543+0000", 2024-09-22T01:03:38.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9157000000000002e-05 2024-09-22T01:03:38.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229717+0000", 2024-09-22T01:03:38.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000174365 2024-09-22T01:03:38.851 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.852 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.852 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51662 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1904640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.852 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:38.852 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.65125095800001, 2024-09-22T01:03:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1004138050000001, 2024-09-22T01:03:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.854 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.854 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51662 2024-09-22T01:03:38.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.854 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:38.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:38.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129659+0000", 2024-09-22T01:03:38.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3699999999999999e-07 2024-09-22T01:03:38.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129663+0000", 2024-09-22T01:03:38.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8199999999999998e-06 2024-09-22T01:03:38.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129681+0000", 2024-09-22T01:03:38.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7972e-05 2024-09-22T01:03:38.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129689+0000", 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-06 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136681+0000", 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069921580000000001 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136692+0000", 2024-09-22T01:03:38.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0977e-05 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229769+0000", 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093076154 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.229926+0000", 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015767099999999999 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230072+0000", 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014609599999999999 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51663 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1933312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.651193577, 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1006063070000001, 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51663 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129716+0000", 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129717+0000", 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8999999999999996e-07 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129720+0000", 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:03:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129744+0000", 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3349e-05 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129751+0000", 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136746+0000", 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069956979999999998 2024-09-22T01:03:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136753+0000", 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.392e-06 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230095+0000", 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.093342187 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230186+0000", 2024-09-22T01:03:38.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1341999999999994e-05 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230322+0000", 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013585499999999999 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51664 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1961984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.651143311, 2024-09-22T01:03:38.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1007743569999999, 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51664 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129766+0000", 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129768+0000", 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129773+0000", 2024-09-22T01:03:38.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129794+0000", 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0956000000000001e-05 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129825+0000", 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1075999999999999e-05 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136808+0000", 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069828620000000003 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136821+0000", 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3393e-05 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230364+0000", 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0935432270000001 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230431+0000", 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6376000000000003e-05 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230541+0000", 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109891 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51665 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1990656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.871 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.651064254, 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1010750030000001, 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51665 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129845+0000", 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129846+0000", 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.133e-06 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129861+0000", 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-05 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129898+0000", 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7045000000000002e-05 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129914+0000", 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5967000000000001e-05 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136864+0000", 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069503910000000002 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136870+0000", 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230599+0000", 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0937293100000001 2024-09-22T01:03:38.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230734+0000", 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013455100000000001 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230920+0000", 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018664500000000001 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51666 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2019328~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.650940005, 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101193173, 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51666 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129970+0000", 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:03:38.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129975+0000", 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2830000000000003e-06 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.129992+0000", 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6977e-05 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130001+0000", 2024-09-22T01:03:38.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.346e-06 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136923+0000", 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069217940000000002 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136936+0000", 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3359e-05 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.230960+0000", 2024-09-22T01:03:38.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094023687 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231041+0000", 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-05 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231163+0000", 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012137 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51667 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2048000~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.650859423, 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1014123929999999, 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51667 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130050+0000", 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130051+0000", 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.13e-06 2024-09-22T01:03:38.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130057+0000", 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130080+0000", 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2782e-05 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130097+0000", 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7180000000000002e-05 2024-09-22T01:03:38.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136978+0000", 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068807620000000003 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.136983+0000", 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231192+0000", 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.094208643 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231281+0000", 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9723999999999997e-05 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231463+0000", 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181156 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51668 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 2076672~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.65079423500001, 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1016154870000001, 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51668 2024-09-22T01:03:38.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130115+0000", 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130116+0000", 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.006e-06 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130122+0000", 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5670000000000002e-06 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130140+0000", 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8448999999999999e-05 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130154+0000", 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.327e-05 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137033+0000", 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068795280000000002 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137044+0000", 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.075e-05 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231510+0000", 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0944666320000001 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231579+0000", 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8620000000000004e-05 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231731+0000", 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015166499999999999 2024-09-22T01:03:38.891 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51779 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1591296~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.650019266, 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101055567, 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.892 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51779 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130890+0000", 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130891+0000", 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130895+0000", 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.907e-06 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130915+0000", 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9715e-05 2024-09-22T01:03:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130923+0000", 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1100000000000003e-06 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140688+0000", 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097652669999999994 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140697+0000", 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6070000000000008e-06 2024-09-22T01:03:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231765+0000", 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091068889 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231836+0000", 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0248999999999998e-05 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231946+0000", 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110163 2024-09-22T01:03:38.896 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51780 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1619968~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.649969481, 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1012166290000001, 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51780 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130940+0000", 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130941+0000", 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1999999999999999e-07 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130944+0000", 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5899999999999999e-06 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130959+0000", 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5063e-05 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.130969+0000", 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4299999999999995e-06 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140719+0000", 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009750257 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140725+0000", 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7660000000000004e-06 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.231976+0000", 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091251618 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232040+0000", 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3116999999999999e-05 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232157+0000", 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117068 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.901 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51781 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1648640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.64988837999999, 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1013338500000001, 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.902 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51781 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131021+0000", 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131022+0000", 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131028+0000", 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6529999999999999e-06 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131049+0000", 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1518999999999998e-05 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131067+0000", 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7476e-05 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140757+0000", 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096901290000000005 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140763+0000", 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1349999999999997e-06 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232190+0000", 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0914267740000001 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232251+0000", 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1063999999999997e-05 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232355+0000", 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010399300000000001 2024-09-22T01:03:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51782 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1677312~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.649818395, 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101464161, 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51782 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131091+0000", 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131092+0000", 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0729999999999999e-06 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131098+0000", 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7559999999999996e-06 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131149+0000", 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0698999999999997e-05 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131157+0000", 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1030000000000002e-06 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140781+0000", 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096246690000000006 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140788+0000", 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4570000000000002e-06 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232388+0000", 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091600055 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232451+0000", 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3268000000000005e-05 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232555+0000", 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104081 2024-09-22T01:03:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51783 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1705984~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.649741031, 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101579933, 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.912 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51783 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131168+0000", 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131169+0000", 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1600000000000001e-07 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131174+0000", 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131195+0000", 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1296000000000001e-05 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131209+0000", 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3403e-05 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140806+0000", 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095974159999999992 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140811+0000", 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2349999999999999e-06 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232588+0000", 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0917768459999999 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232651+0000", 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2673000000000001e-05 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232748+0000", 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7567999999999998e-05 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.916 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4476.0:51784 3.1d 3:bc07d0f6:::rbd_data.117cd666b4d1.0000000000000004:1b [read 1734656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.64968051, 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.101734827, 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4476", 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51784 2024-09-22T01:03:38.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.131229+0000", 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-22T01:03:38.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137670+0000", 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064407960000000004 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137682+0000", 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2679e-05 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137710+0000", 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7066000000000001e-05 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.137729+0000", 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9253e-05 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140832+0000", 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003103462 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:40.140838+0000", 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5180000000000004e-06 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232777+0000", 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.091939682 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232835+0000", 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7299999999999997e-05 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-22T01:01:41.232964+0000", 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129071 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.922 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-22T01:03:38.922 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-22T01:03:38.922 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:38.923 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:03:38.924 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:38.920+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:39.024 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:39.020+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:39.068 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:39.069 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:39.069 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:39.069 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:39.084 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:03:39.126 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:39.124+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:39.226 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:39.224+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:39.258 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:39.258 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:39.258 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:39.258 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:39.258 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:39.272 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:03:39.358 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:39.356+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:39.428 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:39.424+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:39.443 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:39.443 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:39.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55540 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.941359659, 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050273435999999998, 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55540 2024-09-22T01:03:39.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499044+0000", 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499045+0000", 2024-09-22T01:03:39.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-07 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499048+0000", 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499057+0000", 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.346e-06 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499063+0000", 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1800000000000001e-06 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.548991+0000", 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049927465999999997 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549068+0000", 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7569000000000002e-05 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549318+0000", 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000249332 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55541 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.941331816, 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050379488, 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55541 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499072+0000", 2024-09-22T01:03:39.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499073+0000", 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499085+0000", 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2153e-05 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499092+0000", 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8920000000000003e-06 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499097+0000", 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549341+0000", 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050244480000000001 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549390+0000", 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189000000000002e-05 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549452+0000", 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2094999999999994e-05 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55542 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:39.452 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.94129959999999, 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050495911999999997, 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55542 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:39.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499104+0000", 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499105+0000", 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499107+0000", 2024-09-22T01:03:39.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499114+0000", 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000003e-06 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499119+0000", 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1229999999999999e-06 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549482+0000", 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050362756000000002 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549525+0000", 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3319000000000003e-05 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549600+0000", 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4980000000000004e-05 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55543 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.94127786200001, 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050654362000000001, 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55543 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499126+0000", 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499127+0000", 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5499999999999999e-07 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499129+0000", 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499135+0000", 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5309999999999998e-06 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499140+0000", 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7820000000000001e-06 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549630+0000", 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050489806999999998 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549704+0000", 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4344999999999994e-05 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.549781+0000", 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6326000000000001e-05 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55544 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.941258207, 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051110901, 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55544 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:39.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499146+0000", 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000005e-07 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499148+0000", 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499154+0000", 2024-09-22T01:03:39.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.003e-06 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499159+0000", 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550033+0000", 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050873926999999999 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550137+0000", 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010399399999999999 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550257+0000", 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120165 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:55545 3.17 3:e9c984e1:::rbd_data.117cd666b4d1.0000000000000001:1b [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.941227193, 2024-09-22T01:03:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051252114000000001, 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55545 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499177+0000", 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499179+0000", 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:03:39.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499186+0000", 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.499191+0000", 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0200000000000002e-06 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550287+0000", 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051096601999999998 2024-09-22T01:03:39.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550369+0000", 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260000000000005e-05 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:01:41.550429+0000", 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.469 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92264 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21233664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985822982000002, 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050061269999999998, 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92264 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454581+0000", 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454582+0000", 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1260000000000001e-06 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454586+0000", 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0500000000000002e-06 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454600+0000", 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3302000000000001e-05 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454607+0000", 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9240000000000007e-06 2024-09-22T01:03:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504547+0000", 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049939254000000002 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504585+0000", 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8163e-05 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504642+0000", 2024-09-22T01:03:39.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7451000000000003e-05 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92265 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21495808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985783346999995, 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050281826000000002, 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92265 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454621+0000", 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454655+0000", 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3679000000000001e-05 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454670+0000", 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4633e-05 2024-09-22T01:03:39.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454679+0000", 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504693+0000", 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050014014000000002 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504787+0000", 2024-09-22T01:03:39.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4003000000000001e-05 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504902+0000", 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115782 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92266 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 21757952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985695906000004, 2024-09-22T01:03:39.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050436256999999998, 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92266 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454708+0000", 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454709+0000", 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454714+0000", 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6750000000000001e-06 2024-09-22T01:03:39.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454726+0000", 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2277e-05 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454736+0000", 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0105000000000001e-05 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.504938+0000", 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050202064999999997 2024-09-22T01:03:39.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505026+0000", 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7553999999999996e-05 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505144+0000", 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118687 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92267 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22020096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985655687000005, 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050645308, 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92267 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454748+0000", 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454749+0000", 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1800000000000005e-07 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454753+0000", 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-06 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454763+0000", 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0507e-05 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454771+0000", 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8180000000000004e-06 2024-09-22T01:03:39.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505181+0000", 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050409842000000003 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505268+0000", 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7262000000000006e-05 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505394+0000", 2024-09-22T01:03:39.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125451 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92268 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22282240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985619795000005, 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050725765999999999, 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92268 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454784+0000", 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454785+0000", 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454788+0000", 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2090000000000001e-06 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454800+0000", 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454819+0000", 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9315000000000001e-05 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505413+0000", 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050593651000000003 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505453+0000", 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9938000000000003e-05 2024-09-22T01:03:39.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505510+0000", 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7061999999999997e-05 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92269 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22544384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985572146999999, 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050927066999999999, 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92269 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454832+0000", 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454833+0000", 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8800000000000002e-07 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454836+0000", 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.895e-06 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454847+0000", 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0852e-05 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454855+0000", 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0330000000000008e-06 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505556+0000", 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050700470999999997 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505643+0000", 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7102999999999996e-05 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505759+0000", 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116025 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92270 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 22806528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.495 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985537378000004, 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051019875999999999, 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92270 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:39.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454867+0000", 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454868+0000", 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6899999999999996e-07 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454871+0000", 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6040000000000001e-06 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454882+0000", 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0723999999999999e-05 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454890+0000", 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505792+0000", 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050902102999999997 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505830+0000", 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8364000000000001e-05 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505887+0000", 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6443999999999997e-05 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92271 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23068672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985504184999996, 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051233841000000002, 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92271 2024-09-22T01:03:39.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454900+0000", 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454901+0000", 2024-09-22T01:03:39.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000004e-07 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454904+0000", 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4929999999999998e-06 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454915+0000", 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1175000000000001e-05 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454933+0000", 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7333000000000002e-05 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.505927+0000", 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050994549 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506019+0000", 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2214999999999997e-05 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506134+0000", 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114361 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92272 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23330816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985459970999997, 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051300272000000001, 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92272 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454944+0000", 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454945+0000", 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454956+0000", 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454967+0000", 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1277999999999999e-05 2024-09-22T01:03:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454975+0000", 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506152+0000", 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051176658 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506190+0000", 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8318000000000001e-05 2024-09-22T01:03:39.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506244+0000", 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4221000000000001e-05 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92273 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23592960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985417781999999, 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051513775999999997, 2024-09-22T01:03:39.508 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92273 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454986+0000", 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454987+0000", 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.454991+0000", 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0570000000000003e-06 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455002+0000", 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0793e-05 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455009+0000", 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7680000000000008e-06 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506290+0000", 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051280638000000003 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506379+0000", 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9370999999999995e-05 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506500+0000", 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120549 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.512 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92274 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 23855104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985383670000004, 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051603715000000001, 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92274 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455020+0000", 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455021+0000", 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455025+0000", 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.422e-06 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455035+0000", 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.004e-05 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455043+0000", 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0609999999999996e-06 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506518+0000", 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051475386999999997 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506558+0000", 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9570000000000002e-05 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506624+0000", 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6451000000000005e-05 2024-09-22T01:03:39.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92275 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24117248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985350742999998, 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051823622999999999, 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92275 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455053+0000", 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455054+0000", 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455057+0000", 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-06 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455068+0000", 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1068e-05 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455076+0000", 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506669+0000", 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051593282999999997 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506758+0000", 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9380000000000001e-05 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506877+0000", 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011843 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:92276 3.1 3:807ce2f1:::rbd_data.117c77dccaa7.0000000000000004:25 [sparse-read 24379392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e87)", 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 88.985315975000006, 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051904673999999998, 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92276 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455088+0000", 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455089+0000", 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455092+0000", 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.343e-06 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455103+0000", 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0678e-05 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.455110+0000", 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7200000000000006e-06 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506900+0000", 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051789255999999999 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506938+0000", 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8549e-05 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:10.506993+0000", 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4533000000000001e-05 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:151264 3.1b 3:deb34d03:::rbd_object_map.117cf172dcd2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-22T01:03:39.525 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.447127221999999, 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.812141679, 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 151264 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:39.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993277+0000", 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993280+0000", 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8669999999999998e-06 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:39.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993295+0000", 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5235999999999999e-05 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993321+0000", 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5775e-05 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993335+0000", 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4306e-05 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993415+0000", 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9975999999999995e-05 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:05.993426+0000", 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1063000000000001e-05 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978616+0000", 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98518988299999999 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:06.978652+0000", 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5976000000000002e-05 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.803938+0000", 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.82528633799999995 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804052+0000", 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113491 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.804224+0000", 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172375 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805210+0000", 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000986385 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805298+0000", 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7410000000000005e-05 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805343+0000", 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5451999999999998e-05 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:03:07.805418+0000", 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5146000000000002e-05 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:39.532 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:39.534 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:03:39.534 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:39.532+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:39.635 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:39.632+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:39.708 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:39.708 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:39.708 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:39.723 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:03:39.737 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:39.736+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:39.781 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:03:39.838 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:39.836+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:39.849 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:39.850 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:39.850 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:39.850 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:39.850 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:39.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:03:39.938 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:39.936+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:40.001 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:40.001 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1257 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92074456200001, 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054826661999999998, 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1257 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079355+0000", 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079357+0000", 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.714e-06 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079363+0000", 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0279999999999997e-06 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079367+0000", 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.408e-06 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134040+0000", 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054672782000000003 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134109+0000", 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8707999999999993e-05 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134182+0000", 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2694999999999994e-05 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1258 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92072637699999, 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054976851, 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1258 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079373+0000", 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079375+0000", 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.66e-06 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079381+0000", 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999997e-06 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079385+0000", 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2429999999999999e-06 2024-09-22T01:03:40.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134210+0000", 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054824751999999997 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134278+0000", 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7496000000000006e-05 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134350+0000", 2024-09-22T01:03:40.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439000000000002e-05 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1259 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2187264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92070858899999, 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05513995, 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1259 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079391+0000", 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.15e-07 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079393+0000", 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079399+0000", 2024-09-22T01:03:40.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079404+0000", 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134379+0000", 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054975410000000002 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134457+0000", 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7730000000000003e-05 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134531+0000", 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4104999999999996e-05 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1260 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2277376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92069003099999, 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055318415000000003, 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1260 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.016 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079410+0000", 2024-09-22T01:03:40.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079412+0000", 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7239999999999999e-06 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079418+0000", 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079422+0000", 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5290000000000002e-06 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134563+0000", 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055141194999999997 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134650+0000", 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6833e-05 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134728+0000", 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7832999999999998e-05 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1261 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92067197899999, 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055482968000000001, 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1261 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079428+0000", 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0899999999999997e-07 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079430+0000", 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6390000000000001e-06 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079435+0000", 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079440+0000", 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134780+0000", 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055340865000000003 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134858+0000", 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8073999999999998e-05 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134911+0000", 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2102999999999999e-05 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1262 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92065463, 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055684144999999997, 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1262 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079445+0000", 2024-09-22T01:03:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079450+0000", 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1749999999999996e-06 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079457+0000", 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079461+0000", 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4939999999999997e-06 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.134963+0000", 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055501451 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135046+0000", 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3665999999999995e-05 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135129+0000", 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2849000000000006e-05 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1263 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92063267699999, 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055891059999999999, 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.029 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1263 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079467+0000", 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079469+0000", 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6729999999999999e-06 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079475+0000", 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6899999999999997e-06 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079479+0000", 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135184+0000", 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055704803999999997 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135273+0000", 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9005000000000006e-05 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135358+0000", 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5260999999999996e-05 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1264 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.920615142, 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056012019000000003, 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1264 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.034 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079484+0000", 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079485+0000", 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.27e-07 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079486+0000", 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7290000000000001e-06 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079492+0000", 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079502+0000", 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5859999999999994e-06 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135404+0000", 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055902028999999999 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135449+0000", 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4805999999999998e-05 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135496+0000", 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7636999999999998e-05 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1265 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92059135599999, 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056192988999999999, 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1265 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079508+0000", 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079509+0000", 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079510+0000", 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.773e-06 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079516+0000", 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9660000000000001e-06 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079520+0000", 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1960000000000001e-06 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135538+0000", 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056017974999999998 2024-09-22T01:03:40.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135620+0000", 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1627999999999998e-05 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135701+0000", 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-05 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1266 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.920573494, 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056354516, 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1266 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079526+0000", 2024-09-22T01:03:40.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1e-07 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079532+0000", 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0360000000000003e-06 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079539+0000", 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6599999999999998e-06 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079544+0000", 2024-09-22T01:03:40.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6319999999999999e-06 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135779+0000", 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056235779 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135819+0000", 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9833999999999998e-05 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135881+0000", 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1265000000000004e-05 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1267 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2908160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92054625700001, 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056435368999999999, 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1267 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079553+0000", 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079554+0000", 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079556+0000", 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.928e-06 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079562+0000", 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0349999999999998e-06 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079566+0000", 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135905+0000", 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056338895999999999 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135944+0000", 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9292999999999998e-05 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.135989+0000", 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4465999999999997e-05 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1268 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92052755200001, 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056625041000000001, 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1268 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079572+0000", 2024-09-22T01:03:40.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079574+0000", 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419999999999999e-06 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079580+0000", 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079589+0000", 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1130000000000007e-06 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136031+0000", 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442039999999999 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136115+0000", 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3801999999999995e-05 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136197+0000", 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1903000000000005e-05 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1269 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92050405500001, 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056738658999999997, 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1269 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079596+0000", 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079602+0000", 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0549999999999996e-06 2024-09-22T01:03:40.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079608+0000", 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079613+0000", 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136250+0000", 2024-09-22T01:03:40.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056637366000000001 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136290+0000", 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9294e-05 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136334+0000", 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4413000000000003e-05 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1270 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.920480086, 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056918483999999998, 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1270 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079619+0000", 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079620+0000", 2024-09-22T01:03:40.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.46e-07 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079622+0000", 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079628+0000", 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1920000000000002e-06 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079632+0000", 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5040000000000004e-06 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136376+0000", 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056743896000000002 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136455+0000", 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8386000000000005e-05 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136538+0000", 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3325999999999995e-05 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1271 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.92046087099999, 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057127747999999999, 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1271 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.079639+0000", 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080604+0000", 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096492500000000005 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080609+0000", 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.519e-06 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080622+0000", 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2904e-05 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080630+0000", 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7719999999999994e-06 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136587+0000", 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055957570999999998 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136678+0000", 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0826000000000001e-05 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136766+0000", 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8231000000000003e-05 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1272 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.069 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.91945632100001, 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056296712999999998, 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1272 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.080643+0000", 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082943+0000", 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022993380000000002 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082948+0000", 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082960+0000", 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2500999999999999e-05 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.082968+0000", 2024-09-22T01:03:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136839+0000", 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053870973000000003 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136892+0000", 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2961000000000003e-05 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.136940+0000", 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7750999999999997e-05 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:1278 3.19 3:9d8317ee:::rbd_data.117cf538ee01.000000000000001c:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 164.91695036799999, 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054929408999999998, 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1278 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.083149+0000", 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086858+0000", 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00370923 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086864+0000", 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.259e-06 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086876+0000", 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2792999999999999e-05 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.086885+0000", 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.137965+0000", 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051080836999999997 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138034+0000", 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8579999999999997e-05 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:00:55.138079+0000", 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4632000000000002e-05 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104356 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 77.201492208000005, 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060842120999999999, 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104356 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798607+0000", 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798609+0000", 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.694e-06 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798618+0000", 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7239999999999998e-06 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798639+0000", 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1721000000000001e-05 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798650+0000", 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0645000000000001e-05 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798788+0000", 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013750500000000001 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.798941+0000", 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000153208 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.809633+0000", 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01069263 2024-09-22T01:03:40.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859178+0000", 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049544138000000001 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859275+0000", 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7572999999999996e-05 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859372+0000", 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6725999999999995e-05 2024-09-22T01:03:40.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.859449+0000", 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7557000000000003e-05 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:140499 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.942816038000004, 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98072477000000002, 2024-09-22T01:03:40.084 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140499 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057284+0000", 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057286+0000", 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057297+0000", 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026e-05 2024-09-22T01:03:40.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057320+0000", 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3391999999999999e-05 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057331+0000", 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.075e-05 2024-09-22T01:03:40.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057403+0000", 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2566999999999998e-05 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:34.057411+0000", 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2749999999999995e-06 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.037974+0000", 2024-09-22T01:03:40.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98056295800000004 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:35.038008+0000", 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3844999999999999e-05 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:144964 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:03:40.089 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.218560478999997, 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95820099800000003, 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144964 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781539+0000", 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781542+0000", 2024-09-22T01:03:40.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4899999999999999e-06 2024-09-22T01:03:40.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781553+0000", 2024-09-22T01:03:40.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1687e-05 2024-09-22T01:03:40.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781583+0000", 2024-09-22T01:03:40.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0173999999999999e-05 2024-09-22T01:03:40.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781597+0000", 2024-09-22T01:03:40.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3172e-05 2024-09-22T01:03:40.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781681+0000", 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4810999999999998e-05 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:38.781692+0000", 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0662000000000001e-05 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739463+0000", 2024-09-22T01:03:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95777067000000005 2024-09-22T01:03:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739603+0000", 2024-09-22T01:03:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013200000000001 2024-09-22T01:03:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:39.739740+0000", 2024-09-22T01:03:40.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001372 2024-09-22T01:03:40.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.098 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.098 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.098 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.098 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:40.100 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:03:40.100 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-09-22T01:03:40.096+0000 7f1f7f33c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26647) UID: 0 2024-09-22T01:03:40.200 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:40.200+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:40.264 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:40.264 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:40.264 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:40.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:03:40.301 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:40.300+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:40.402 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:40.400+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:40.405 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:40.405 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:40.405 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-22T01:03:40.405 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-22T01:03:40.405 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:40.420 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:03:40.504 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:40.500+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:40.547 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:40.547 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3.262a7b13 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 96.772950891999997, 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366911223, 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771092+0000", 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771094+0000", 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0930000000000001e-06 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771103+0000", 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0629999999999995e-06 2024-09-22T01:03:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771127+0000", 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3924e-05 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771147+0000", 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771214+0000", 2024-09-22T01:03:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7348999999999995e-05 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771220+0000", 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3709999999999996e-06 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.137980+0000", 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366760011 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138003+0000", 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2823000000000002e-05 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3.c7ddc9b3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 96.772863146999995, 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.367176741, 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:03:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771180+0000", 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771181+0000", 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8700000000000005e-07 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771231+0000", 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771247+0000", 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5858999999999998e-05 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771259+0000", 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2814e-05 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771320+0000", 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0393999999999998e-05 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.771325+0000", 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138340+0000", 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3670150409999999 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138356+0000", 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6782999999999998e-05 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86715 3.13 3:c8de5464:::rbd_data.117c2270b223.000000000000001e:head [sparse-read 0~4194304] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 96.771941448999996, 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366676402, 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:03:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86715 2024-09-22T01:03:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:03:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772101+0000", 2024-09-22T01:03:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772103+0000", 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9929999999999998e-06 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772113+0000", 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.499e-06 2024-09-22T01:03:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772138+0000", 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4743999999999999e-05 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772151+0000", 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3189e-05 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772415+0000", 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026382499999999998 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772427+0000", 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2103000000000001e-05 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138485+0000", 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366057984 2024-09-22T01:03:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138778+0000", 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029306500000000001 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:86741 3.13 3:cd93bbe3:::rbd_data.117c2270b223.0000000000000038:head [sparse-read 0~3145728] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-22T01:03:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 96.771865333999997, 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.366907898, 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 86741 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772178+0000", 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5600000000000005e-07 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772183+0000", 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772195+0000", 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1533e-05 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772206+0000", 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1647e-05 2024-09-22T01:03:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772493+0000", 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028702699999999997 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:03.772511+0000", 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7609999999999999e-05 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.138879+0000", 2024-09-22T01:03:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3663676759999999 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:05.139085+0000", 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020690899999999999 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:90676 3.9 3:963e58c3:::rbd_data.117c77dccaa7.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:03:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 91.001933682000001, 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093363432999999996, 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 90676 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542109+0000", 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542112+0000", 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9730000000000002e-06 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542126+0000", 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3543999999999999e-05 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542161+0000", 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4975000000000003e-05 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542185+0000", 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4613999999999998e-05 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542321+0000", 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013617999999999999 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.542562+0000", 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024014899999999999 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.556142+0000", 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013580126 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635270+0000", 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079128575000000007 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635299+0000", 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8996000000000002e-05 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635391+0000", 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1712000000000006e-05 2024-09-22T01:03:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:09.635473+0000", 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1588999999999994e-05 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:104388 3.0 3:00c9df4b:::rbd_data.117cde125e21.000000000000000a:head [write 9699485~40067] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:03:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 77.611311637, 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896004440000002, 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104388 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:03:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932731+0000", 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932733+0000", 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1780000000000002e-06 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932760+0000", 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6174e-05 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932775+0000", 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5603e-05 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932787+0000", 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2155e-05 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932873+0000", 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5391999999999998e-05 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:22.932887+0000", 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3915e-05 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930412+0000", 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.99752547400000002 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:23.930446+0000", 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3841999999999999e-05 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.019619+0000", 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.089172569 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020035+0000", 2024-09-22T01:03:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041613899999999999 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.020214+0000", 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017929999999999999 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.021438+0000", 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012240440000000001 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:03:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022156+0000", 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071828699999999996 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022231+0000", 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420999999999998e-05 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:25.022332+0000", 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100951 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106194 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 778240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.160469040999999, 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091629434999999995, 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106194 2024-09-22T01:03:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383574+0000", 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383577+0000", 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7549999999999999e-06 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383585+0000", 2024-09-22T01:03:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4209999999999998e-06 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383599+0000", 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4599e-05 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.474958+0000", 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091358710999999995 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475039+0000", 2024-09-22T01:03:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1075999999999994e-05 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475203+0000", 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016427600000000001 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106195 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.160434198000004, 2024-09-22T01:03:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091906246999999996, 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106195 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383609+0000", 2024-09-22T01:03:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383612+0000", 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383620+0000", 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0600000000000008e-06 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383626+0000", 2024-09-22T01:03:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8529999999999997e-06 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475245+0000", 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091618623999999996 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475326+0000", 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1166000000000002e-05 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475515+0000", 2024-09-22T01:03:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018890399999999999 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106196 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.160408175000001, 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092159059000000002, 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106196 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383635+0000", 2024-09-22T01:03:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-22T01:03:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383645+0000", 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7119999999999995e-06 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383653+0000", 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.807e-06 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383658+0000", 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.04e-06 2024-09-22T01:03:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475547+0000", 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091889557999999996 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475630+0000", 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3114000000000004e-05 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475794+0000", 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016325199999999999 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106197 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 821248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.160375474999995, 2024-09-22T01:03:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092422578000000005, 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106197 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383667+0000", 2024-09-22T01:03:40.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383668+0000", 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.63e-07 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383677+0000", 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8859999999999993e-06 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383685+0000", 2024-09-22T01:03:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5609999999999999e-06 2024-09-22T01:03:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383690+0000", 2024-09-22T01:03:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1499999999999998e-06 2024-09-22T01:03:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475825+0000", 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092134439999999998 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.475917+0000", 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2559000000000006e-05 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476090+0000", 2024-09-22T01:03:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017251899999999999 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106198 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.160342314000005, 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092626416000000003, 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106198 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.383701+0000", 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384316+0000", 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061580100000000002 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384323+0000", 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384338+0000", 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5126999999999999e-05 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384348+0000", 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8479999999999996e-06 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476109+0000", 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091760280999999999 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476178+0000", 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9865999999999998e-05 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476327+0000", 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014848199999999999 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106199 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.159677896999995, 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092274072999999998, 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106199 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.384365+0000", 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385301+0000", 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093565 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385312+0000", 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1589e-05 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385342+0000", 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9743999999999998e-05 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385361+0000", 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9219999999999999e-05 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476358+0000", 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090996442999999996 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476438+0000", 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9977999999999999e-05 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476639+0000", 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000201449 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106200 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.158649909999994, 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091589137000000001, 2024-09-22T01:03:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106200 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385393+0000", 2024-09-22T01:03:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385394+0000", 2024-09-22T01:03:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1599999999999999e-06 2024-09-22T01:03:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385401+0000", 2024-09-22T01:03:40.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9090000000000003e-06 2024-09-22T01:03:40.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385420+0000", 2024-09-22T01:03:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-22T01:03:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385434+0000", 2024-09-22T01:03:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4068e-05 2024-09-22T01:03:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476677+0000", 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091242904999999999 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476760+0000", 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3783999999999996e-05 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.476982+0000", 2024-09-22T01:03:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022163200000000001 2024-09-22T01:03:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106201 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:03:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.158428424999997, 2024-09-22T01:03:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091715669, 2024-09-22T01:03:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106201 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385614+0000", 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385616+0000", 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.277e-06 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385624+0000", 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7910000000000005e-06 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385646+0000", 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2251000000000001e-05 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385680+0000", 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3933000000000002e-05 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477014+0000", 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091334175000000004 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477096+0000", 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2361999999999995e-05 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477330+0000", 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023388 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106202 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.158340007999996, 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091937313000000007, 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106202 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:03:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385703+0000", 2024-09-22T01:03:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385835+0000", 2024-09-22T01:03:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013252699999999999 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385844+0000", 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1599999999999998e-06 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385864+0000", 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0579000000000001e-05 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385879+0000", 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4871e-05 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477362+0000", 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091483156999999996 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477452+0000", 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9868000000000001e-05 2024-09-22T01:03:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477640+0000", 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018815100000000001 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106203 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.158121175000005, 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092020950000000004, 2024-09-22T01:03:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106203 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.385922+0000", 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386095+0000", 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017321200000000001 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386104+0000", 2024-09-22T01:03:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5109999999999992e-06 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386129+0000", 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4785000000000001e-05 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386145+0000", 2024-09-22T01:03:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6191000000000001e-05 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477675+0000", 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091529570000000005 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477762+0000", 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6861999999999996e-05 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477943+0000", 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180819 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106204 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.157870736999996, 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092064394999999993, 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106204 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386172+0000", 2024-09-22T01:03:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386406+0000", 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023343699999999999 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386412+0000", 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386425+0000", 2024-09-22T01:03:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2719e-05 2024-09-22T01:03:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386433+0000", 2024-09-22T01:03:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2810000000000008e-06 2024-09-22T01:03:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.477976+0000", 2024-09-22T01:03:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091542739999999997 2024-09-22T01:03:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478069+0000", 2024-09-22T01:03:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2971000000000001e-05 2024-09-22T01:03:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478237+0000", 2024-09-22T01:03:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167949 2024-09-22T01:03:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106205 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:03:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.157595069999999, 2024-09-22T01:03:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092018127000000005, 2024-09-22T01:03:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106205 2024-09-22T01:03:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:03:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.386448+0000", 2024-09-22T01:03:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387017+0000", 2024-09-22T01:03:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056957000000000004 2024-09-22T01:03:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387024+0000", 2024-09-22T01:03:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-22T01:03:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387035+0000", 2024-09-22T01:03:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1187e-05 2024-09-22T01:03:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387045+0000", 2024-09-22T01:03:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0750000000000004e-06 2024-09-22T01:03:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478259+0000", 2024-09-22T01:03:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091214394000000004 2024-09-22T01:03:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478311+0000", 2024-09-22T01:03:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1984000000000002e-05 2024-09-22T01:03:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478466+0000", 2024-09-22T01:03:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015504200000000001 2024-09-22T01:03:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106206 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:03:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.156979684000007, 2024-09-22T01:03:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091679640000000007, 2024-09-22T01:03:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106206 2024-09-22T01:03:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:03:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387063+0000", 2024-09-22T01:03:40.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387533+0000", 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047020100000000001 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387539+0000", 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8200000000000002e-06 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387552+0000", 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2865e-05 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387560+0000", 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5499999999999997e-06 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478507+0000", 2024-09-22T01:03:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090947741999999998 2024-09-22T01:03:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478599+0000", 2024-09-22T01:03:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1181000000000007e-05 2024-09-22T01:03:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.478743+0000", 2024-09-22T01:03:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014428100000000001 2024-09-22T01:03:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4476.0:106208 3.1a 3:598cb1fc:::rbd_data.117c23b3fc7.0000000000000000:2e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-22T01:03:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:03:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.156430881000006, 2024-09-22T01:03:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091664238999999995, 2024-09-22T01:03:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-22T01:03:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-22T01:03:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-22T01:03:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-22T01:03:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4476", 2024-09-22T01:03:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/4027601204", 2024-09-22T01:03:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106208 2024-09-22T01:03:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-22T01:03:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-22T01:03:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:03:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-22T01:03:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.387612+0000", 2024-09-22T01:03:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-22T01:03:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391033+0000", 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00342144 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391040+0000", 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8140000000000003e-06 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391054+0000", 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4089e-05 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.391063+0000", 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3809999999999999e-06 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479014+0000", 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087951647999999993 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479099+0000", 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4859999999999997e-05 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-22T01:02:29.479276+0000", 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017700699999999999 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-22T01:03:40.668 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:40.670 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:03:40.670 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:40.668+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:40.742 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:03:40.806 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:40.804+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:40.848 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:03:40.848 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-22T01:03:40.848 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-22T01:03:40.848 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:03:40.864 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:03:40.874 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:03:40.876 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:40.872+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:40.926 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-22T01:03:40.926 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:03:40.943 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-22T01:03:40.943 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:03:40.961 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-22T01:03:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:03:40.972 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:40.968+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:40.978 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-22T01:03:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:03:41.000 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-22T01:03:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:03:41.005 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:41.005 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:41.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:03:41.023 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-22T01:03:41.023 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:03:41.051 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-22T01:03:41.109 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:03:41.109 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-22T01:03:41.110 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:03:41.111 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:41.108+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:41.148 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:41.148 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:41.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:03:41.215 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-09-22T01:03:41.212+0000 7f29005d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26649) UID: 0 2024-09-22T01:03:41.295 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:41.295 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:41.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:03:41.316 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:41.315+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:41.396 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7, 1, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-22T01:03:41.396 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T01:03:41.442 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-22T01:03:41.439+0000 7f97d1465640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26416) UID: 0 2024-09-22T01:03:41.477 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:41.477 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:41.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:03:41.519 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:41.515+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:41.609 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:41.609 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:41.622 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-22T01:03:41.615+0000 7fab3ff8a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26418) UID: 0 2024-09-22T01:03:41.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:03:41.720 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-22T01:03:41.715+0000 7f18081c7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26422) UID: 0 2024-09-22T01:03:41.746 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:41.746 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:41.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:03:41.821 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:41.815+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:41.879 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:41.879 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T01:03:41.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:03:41.909 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T01:03:41.954 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-22T01:03:41.951+0000 7fc1ffed2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26420) UID: 0 2024-09-22T01:03:42.023 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-22T01:03:42.020+0000 7fa37e7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35132) UID: 0 2024-09-22T01:03:42.025 INFO:teuthology.orchestra.run.smithi002.stdout:All operations completed A-OK! 2024-09-22T01:03:42.028 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:42.028 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:42.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:03:42.075 DEBUG:teuthology.parallel:result is None 2024-09-22T01:03:42.075 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-22T01:03:42.085 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-22T01:03:42.144 INFO:tasks.thrashosds:joining thrashosds 2024-09-22T01:03:42.272 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:42.272 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:42.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:03:42.410 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:42.410 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:42.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:03:42.549 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:42.550 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:42.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:03:42.687 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:42.687 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:42.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:03:42.751 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T01:03:42.779 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T01:03:42.826 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:42.826 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:42.840 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:03:42.909 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:03:42.964 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-22T01:03:42.964 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:03:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T01:03:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-22T01:03:43.914 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-22T01:03:43.918 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:03:44.895 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.5 to 1 (10000) 2024-09-22T01:03:44.895 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-22T01:03:44.923 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-22T01:03:45.917 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.6 to 1 (10000) 2024-09-22T01:03:45.940 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-22T01:03:46.913 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.7 to 1 (10000) 2024-09-22T01:03:46.940 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T01:03:47.746 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T01:03:47.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-22T01:03:48.931 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.4 to 1 (10000) 2024-09-22T01:03:48.957 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T01:03:48.958 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:03:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:03:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:03:49.289 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-22T01:03:48.933199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20067,"ondisk_log_size":20067,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":376856,"kb_used_data":17952,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754597864,"statfs":{"total":773094113280,"available":772708212736,"internally_reserved":0,"allocated":18382848,"data_stored":9488378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-71804757,"num_objects":-87,"num_object_clones":-31,"num_object_copies":-174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-87,"num_whiteouts":-21,"num_read":252,"num_read_kb":218,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.036986"},"pg_stats":[{"pgid":"3.1f","version":"176'532","reported_seq":7167,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208575+0000","last_change":"2024-09-22T01:03:41.888720+0000","last_active":"2024-09-22T01:03:45.208575+0000","last_peered":"2024-09-22T01:03:45.208575+0000","last_clean":"2024-09-22T01:03:45.208575+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:03:45.208575+0000","last_undegraded":"2024-09-22T01:03:45.208575+0000","last_fullsized":"2024-09-22T01:03:45.208575+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:31.136135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1e","version":"176'811","reported_seq":3831,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208220+0000","last_change":"2024-09-22T01:03:41.885281+0000","last_active":"2024-09-22T01:03:45.208220+0000","last_peered":"2024-09-22T01:03:45.208220+0000","last_clean":"2024-09-22T01:03:45.208220+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:03:45.208220+0000","last_undegraded":"2024-09-22T01:03:45.208220+0000","last_fullsized":"2024-09-22T01:03:45.208220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:49.748988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1d","version":"175'502","reported_seq":5322,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914876+0000","last_change":"2024-09-22T01:03:41.907575+0000","last_active":"2024-09-22T01:03:45.914876+0000","last_peered":"2024-09-22T01:03:45.914876+0000","last_clean":"2024-09-22T01:03:45.914876+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:03:45.914876+0000","last_undegraded":"2024-09-22T01:03:45.914876+0000","last_fullsized":"2024-09-22T01:03:45.914876+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:35:17.317230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1c","version":"176'1126","reported_seq":10706,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914552+0000","last_change":"2024-09-22T01:03:41.905727+0000","last_active":"2024-09-22T01:03:45.914552+0000","last_peered":"2024-09-22T01:03:45.914552+0000","last_clean":"2024-09-22T01:03:45.914552+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:03:45.914552+0000","last_undegraded":"2024-09-22T01:03:45.914552+0000","last_fullsized":"2024-09-22T01:03:45.914552+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:36.367987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1b","version":"175'1098","reported_seq":3104,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208470+0000","last_change":"2024-09-22T01:03:41.889650+0000","last_active":"2024-09-22T01:03:45.208470+0000","last_peered":"2024-09-22T01:03:45.208470+0000","last_clean":"2024-09-22T01:03:45.208470+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:03:45.208470+0000","last_undegraded":"2024-09-22T01:03:45.208470+0000","last_fullsized":"2024-09-22T01:03:45.208470+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:09.100865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1a","version":"175'484","reported_seq":12360,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898472+0000","last_change":"2024-09-22T01:03:41.912630+0000","last_active":"2024-09-22T01:03:44.898472+0000","last_peered":"2024-09-22T01:03:44.898472+0000","last_clean":"2024-09-22T01:03:44.898472+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:03:44.898472+0000","last_undegraded":"2024-09-22T01:03:44.898472+0000","last_fullsized":"2024-09-22T01:03:44.898472+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:30:44.630235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.19","version":"176'401","reported_seq":10051,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898276+0000","last_change":"2024-09-22T01:03:41.888321+0000","last_active":"2024-09-22T01:03:44.898276+0000","last_peered":"2024-09-22T01:03:44.898276+0000","last_clean":"2024-09-22T01:03:44.898276+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:03:44.898276+0000","last_undegraded":"2024-09-22T01:03:44.898276+0000","last_fullsized":"2024-09-22T01:03:44.898276+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:18.365366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.18","version":"176'793","reported_seq":6603,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.207609+0000","last_change":"2024-09-22T01:03:41.882274+0000","last_active":"2024-09-22T01:03:45.207609+0000","last_peered":"2024-09-22T01:03:45.207609+0000","last_clean":"2024-09-22T01:03:45.207609+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:03:45.207609+0000","last_undegraded":"2024-09-22T01:03:45.207609+0000","last_fullsized":"2024-09-22T01:03:45.207609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:12.860843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.17","version":"175'563","reported_seq":4371,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210578+0000","last_change":"2024-09-22T01:03:41.894428+0000","last_active":"2024-09-22T01:03:45.210578+0000","last_peered":"2024-09-22T01:03:45.210578+0000","last_clean":"2024-09-22T01:03:45.210578+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:03:45.210578+0000","last_undegraded":"2024-09-22T01:03:45.210578+0000","last_fullsized":"2024-09-22T01:03:45.210578+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:55.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.16","version":"175'369","reported_seq":2246,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914752+0000","last_change":"2024-09-22T01:03:41.907362+0000","last_active":"2024-09-22T01:03:45.914752+0000","last_peered":"2024-09-22T01:03:45.914752+0000","last_clean":"2024-09-22T01:03:45.914752+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:03:45.914752+0000","last_undegraded":"2024-09-22T01:03:45.914752+0000","last_fullsized":"2024-09-22T01:03:45.914752+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:08.940607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.15","version":"176'687","reported_seq":6846,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.209418+0000","last_change":"2024-09-22T01:03:41.999115+0000","last_active":"2024-09-22T01:03:45.209418+0000","last_peered":"2024-09-22T01:03:45.209418+0000","last_clean":"2024-09-22T01:03:45.209418+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:03:45.209418+0000","last_undegraded":"2024-09-22T01:03:45.209418+0000","last_fullsized":"2024-09-22T01:03:45.209418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:33.921063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.14","version":"176'563","reported_seq":13946,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898883+0000","last_change":"2024-09-22T01:03:41.912104+0000","last_active":"2024-09-22T01:03:44.898883+0000","last_peered":"2024-09-22T01:03:44.898883+0000","last_clean":"2024-09-22T01:03:44.898883+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:03:44.898883+0000","last_undegraded":"2024-09-22T01:03:44.898883+0000","last_fullsized":"2024-09-22T01:03:44.898883+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:42.625118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":336,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210652+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:03:45.210652+0000","last_peered":"2024-09-22T01:03:45.210652+0000","last_clean":"2024-09-22T01:03:45.210652+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:03:45.210652+0000","last_undegraded":"2024-09-22T01:03:45.210652+0000","last_fullsized":"2024-09-22T01:03:45.210652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"176'605","reported_seq":3282,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208472+0000","last_change":"2024-09-22T01:03:41.887491+0000","last_active":"2024-09-22T01:03:45.208472+0000","last_peered":"2024-09-22T01:03:45.208472+0000","last_clean":"2024-09-22T01:03:45.208472+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:03:45.208472+0000","last_undegraded":"2024-09-22T01:03:45.208472+0000","last_fullsized":"2024-09-22T01:03:45.208472+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:31.037935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898529+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:03:44.898529+0000","last_peered":"2024-09-22T01:03:44.898529+0000","last_clean":"2024-09-22T01:03:44.898529+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:03:44.898529+0000","last_undegraded":"2024-09-22T01:03:44.898529+0000","last_fullsized":"2024-09-22T01:03:44.898529+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'565","reported_seq":3615,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208332+0000","last_change":"2024-09-22T01:03:41.886088+0000","last_active":"2024-09-22T01:03:45.208332+0000","last_peered":"2024-09-22T01:03:45.208332+0000","last_clean":"2024-09-22T01:03:45.208332+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:03:45.208332+0000","last_undegraded":"2024-09-22T01:03:45.208332+0000","last_fullsized":"2024-09-22T01:03:45.208332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:49.543414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":336,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210051+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:03:45.210051+0000","last_peered":"2024-09-22T01:03:45.210051+0000","last_clean":"2024-09-22T01:03:45.210051+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:03:45.210051+0000","last_undegraded":"2024-09-22T01:03:45.210051+0000","last_fullsized":"2024-09-22T01:03:45.210051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'444","reported_seq":4478,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898206+0000","last_change":"2024-09-22T01:03:41.883039+0000","last_active":"2024-09-22T01:03:44.898206+0000","last_peered":"2024-09-22T01:03:44.898206+0000","last_clean":"2024-09-22T01:03:44.898206+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:03:44.898206+0000","last_undegraded":"2024-09-22T01:03:44.898206+0000","last_fullsized":"2024-09-22T01:03:44.898206+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:52.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898178+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:03:44.898178+0000","last_peered":"2024-09-22T01:03:44.898178+0000","last_clean":"2024-09-22T01:03:44.898178+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:03:44.898178+0000","last_undegraded":"2024-09-22T01:03:44.898178+0000","last_fullsized":"2024-09-22T01:03:44.898178+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'680","reported_seq":4649,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.915163+0000","last_change":"2024-09-22T01:03:41.906860+0000","last_active":"2024-09-22T01:03:45.915163+0000","last_peered":"2024-09-22T01:03:45.915163+0000","last_clean":"2024-09-22T01:03:45.915163+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:03:45.915163+0000","last_undegraded":"2024-09-22T01:03:45.915163+0000","last_fullsized":"2024-09-22T01:03:45.915163+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:43.965472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.3","version":"176'736","reported_seq":3057,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898748+0000","last_change":"2024-09-22T01:03:41.913492+0000","last_active":"2024-09-22T01:03:44.898748+0000","last_peered":"2024-09-22T01:03:44.898748+0000","last_clean":"2024-09-22T01:03:44.898748+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:03:44.898748+0000","last_undegraded":"2024-09-22T01:03:44.898748+0000","last_fullsized":"2024-09-22T01:03:44.898748+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:27.079591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.2","version":"21'2","reported_seq":311,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914485+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:03:45.914485+0000","last_peered":"2024-09-22T01:03:45.914485+0000","last_clean":"2024-09-22T01:03:45.914485+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:03:45.914485+0000","last_undegraded":"2024-09-22T01:03:45.914485+0000","last_fullsized":"2024-09-22T01:03:45.914485+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'1653","reported_seq":3909,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898087+0000","last_change":"2024-09-22T01:03:41.883040+0000","last_active":"2024-09-22T01:03:44.898087+0000","last_peered":"2024-09-22T01:03:44.898087+0000","last_clean":"2024-09-22T01:03:44.898087+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:03:44.898087+0000","last_undegraded":"2024-09-22T01:03:44.898087+0000","last_fullsized":"2024-09-22T01:03:44.898087+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1653,"log_dups_size":0,"ondisk_log_size":1653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:43.607282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":336,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.897847+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:03:44.897847+0000","last_peered":"2024-09-22T01:03:44.897847+0000","last_clean":"2024-09-22T01:03:44.897847+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:03:44.897847+0000","last_undegraded":"2024-09-22T01:03:44.897847+0000","last_fullsized":"2024-09-22T01:03:44.897847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'388","reported_seq":3128,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.207994+0000","last_change":"2024-09-22T01:03:41.886773+0000","last_active":"2024-09-22T01:03:45.207994+0000","last_peered":"2024-09-22T01:03:45.207994+0000","last_clean":"2024-09-22T01:03:45.207994+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:03:45.207994+0000","last_undegraded":"2024-09-22T01:03:45.207994+0000","last_fullsized":"2024-09-22T01:03:45.207994+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:39.279522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":351,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208871+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:03:45.208871+0000","last_peered":"2024-09-22T01:03:45.208871+0000","last_clean":"2024-09-22T01:03:45.208871+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:03:45.208871+0000","last_undegraded":"2024-09-22T01:03:45.208871+0000","last_fullsized":"2024-09-22T01:03:45.208871+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'392","reported_seq":2363,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.207870+0000","last_change":"2024-09-22T01:03:41.882465+0000","last_active":"2024-09-22T01:03:45.207870+0000","last_peered":"2024-09-22T01:03:45.207870+0000","last_clean":"2024-09-22T01:03:45.207870+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:03:45.207870+0000","last_undegraded":"2024-09-22T01:03:45.207870+0000","last_fullsized":"2024-09-22T01:03:45.207870+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:12.779986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.3","version":"19'1","reported_seq":292,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914505+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:03:45.914505+0000","last_peered":"2024-09-22T01:03:45.914505+0000","last_clean":"2024-09-22T01:03:45.914505+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:03:45.914505+0000","last_undegraded":"2024-09-22T01:03:45.914505+0000","last_fullsized":"2024-09-22T01:03:45.914505+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":487,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208996+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:03:45.208996+0000","last_peered":"2024-09-22T01:03:45.208996+0000","last_clean":"2024-09-22T01:03:45.208996+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:03:45.208996+0000","last_undegraded":"2024-09-22T01:03:45.208996+0000","last_fullsized":"2024-09-22T01:03:45.208996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'487","reported_seq":10756,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.207692+0000","last_change":"2024-09-22T01:03:41.883318+0000","last_active":"2024-09-22T01:03:45.207692+0000","last_peered":"2024-09-22T01:03:45.207692+0000","last_clean":"2024-09-22T01:03:45.207692+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:03:45.207692+0000","last_undegraded":"2024-09-22T01:03:45.207692+0000","last_fullsized":"2024-09-22T01:03:45.207692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:37.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.9","version":"175'603","reported_seq":2550,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898657+0000","last_change":"2024-09-22T01:03:41.912974+0000","last_active":"2024-09-22T01:03:44.898657+0000","last_peered":"2024-09-22T01:03:44.898657+0000","last_clean":"2024-09-22T01:03:44.898657+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:03:44.898657+0000","last_undegraded":"2024-09-22T01:03:44.898657+0000","last_fullsized":"2024-09-22T01:03:44.898657+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:19.230227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.a","version":"176'801","reported_seq":16424,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.209712+0000","last_change":"2024-09-22T01:03:41.893384+0000","last_active":"2024-09-22T01:03:45.209712+0000","last_peered":"2024-09-22T01:03:45.209712+0000","last_clean":"2024-09-22T01:03:45.209712+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:03:45.209712+0000","last_undegraded":"2024-09-22T01:03:45.209712+0000","last_fullsized":"2024-09-22T01:03:45.209712+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:18.880496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.b","version":"175'701","reported_seq":2951,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208114+0000","last_change":"2024-09-22T01:03:41.889194+0000","last_active":"2024-09-22T01:03:45.208114+0000","last_peered":"2024-09-22T01:03:45.208114+0000","last_clean":"2024-09-22T01:03:45.208114+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:03:45.208114+0000","last_undegraded":"2024-09-22T01:03:45.208114+0000","last_fullsized":"2024-09-22T01:03:45.208114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:52.453227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.c","version":"176'418","reported_seq":1553,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914616+0000","last_change":"2024-09-22T01:03:41.906335+0000","last_active":"2024-09-22T01:03:45.914616+0000","last_peered":"2024-09-22T01:03:45.914616+0000","last_clean":"2024-09-22T01:03:45.914616+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:03:45.914616+0000","last_undegraded":"2024-09-22T01:03:45.914616+0000","last_fullsized":"2024-09-22T01:03:45.914616+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:26.787651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.d","version":"175'759","reported_seq":5987,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.209765+0000","last_change":"2024-09-22T01:03:41.999171+0000","last_active":"2024-09-22T01:03:45.209765+0000","last_peered":"2024-09-22T01:03:45.209765+0000","last_clean":"2024-09-22T01:03:45.209765+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:03:45.209765+0000","last_undegraded":"2024-09-22T01:03:45.209765+0000","last_fullsized":"2024-09-22T01:03:45.209765+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:25.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.e","version":"176'486","reported_seq":5181,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210341+0000","last_change":"2024-09-22T01:03:42.000884+0000","last_active":"2024-09-22T01:03:45.210341+0000","last_peered":"2024-09-22T01:03:45.210341+0000","last_clean":"2024-09-22T01:03:45.210341+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:03:45.210341+0000","last_undegraded":"2024-09-22T01:03:45.210341+0000","last_fullsized":"2024-09-22T01:03:45.210341+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:01.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.f","version":"174'399","reported_seq":5196,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210682+0000","last_change":"2024-09-22T01:03:42.001967+0000","last_active":"2024-09-22T01:03:45.210682+0000","last_peered":"2024-09-22T01:03:45.210682+0000","last_clean":"2024-09-22T01:03:45.210682+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:03:45.210682+0000","last_undegraded":"2024-09-22T01:03:45.210682+0000","last_fullsized":"2024-09-22T01:03:45.210682+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:14.538603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.10","version":"176'462","reported_seq":3372,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.209612+0000","last_change":"2024-09-22T01:03:41.893276+0000","last_active":"2024-09-22T01:03:45.209612+0000","last_peered":"2024-09-22T01:03:45.209612+0000","last_clean":"2024-09-22T01:03:45.209612+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:03:45.209612+0000","last_undegraded":"2024-09-22T01:03:45.209612+0000","last_fullsized":"2024-09-22T01:03:45.209612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:32.260241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.11","version":"175'680","reported_seq":3027,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210054+0000","last_change":"2024-09-22T01:03:41.997899+0000","last_active":"2024-09-22T01:03:45.210054+0000","last_peered":"2024-09-22T01:03:45.210054+0000","last_clean":"2024-09-22T01:03:45.210054+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:03:45.210054+0000","last_undegraded":"2024-09-22T01:03:45.210054+0000","last_fullsized":"2024-09-22T01:03:45.210054+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.873636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.12","version":"174'445","reported_seq":3232,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208585+0000","last_change":"2024-09-22T01:03:41.888133+0000","last_active":"2024-09-22T01:03:45.208585+0000","last_peered":"2024-09-22T01:03:45.208585+0000","last_clean":"2024-09-22T01:03:45.208585+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:03:45.208585+0000","last_undegraded":"2024-09-22T01:03:45.208585+0000","last_fullsized":"2024-09-22T01:03:45.208585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:14.342554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.13","version":"175'314","reported_seq":6862,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208000+0000","last_change":"2024-09-22T01:03:41.881304+0000","last_active":"2024-09-22T01:03:45.208000+0000","last_peered":"2024-09-22T01:03:45.208000+0000","last_clean":"2024-09-22T01:03:45.208000+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:03:45.208000+0000","last_undegraded":"2024-09-22T01:03:45.208000+0000","last_fullsized":"2024-09-22T01:03:45.208000+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.277966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19947,"ondisk_log_size":19947,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2150400,"data_stored":1039927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299421,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30236,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341604,"statfs":{"total":96636764160,"available":96605802496,"internally_reserved":0,"allocated":2166784,"data_stored":1045871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60240,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311600,"statfs":{"total":96636764160,"available":96575078400,"internally_reserved":0,"allocated":2744320,"data_stored":1630295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2142208,"data_stored":1034021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47172,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324668,"statfs":{"total":96636764160,"available":96588460032,"internally_reserved":0,"allocated":2138112,"data_stored":1034002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51328,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320512,"statfs":{"total":96636764160,"available":96584204288,"internally_reserved":0,"allocated":2138112,"data_stored":1034002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51344,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320496,"statfs":{"total":96636764160,"available":96584187904,"internally_reserved":0,"allocated":2158592,"data_stored":1039965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47760,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324080,"statfs":{"total":96636764160,"available":96587857920,"internally_reserved":0,"allocated":2744320,"data_stored":1630295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:03:49.291 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:03:49.603 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:03:49.604 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:03:49.623 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-22T01:03:48.933199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20067,"ondisk_log_size":20067,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":376856,"kb_used_data":17952,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754597864,"statfs":{"total":773094113280,"available":772708212736,"internally_reserved":0,"allocated":18382848,"data_stored":9488378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-71804757,"num_objects":-87,"num_object_clones":-31,"num_object_copies":-174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-87,"num_whiteouts":-21,"num_read":252,"num_read_kb":218,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.036986"},"pg_stats":[{"pgid":"3.1f","version":"176'532","reported_seq":7167,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208575+0000","last_change":"2024-09-22T01:03:41.888720+0000","last_active":"2024-09-22T01:03:45.208575+0000","last_peered":"2024-09-22T01:03:45.208575+0000","last_clean":"2024-09-22T01:03:45.208575+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:03:45.208575+0000","last_undegraded":"2024-09-22T01:03:45.208575+0000","last_fullsized":"2024-09-22T01:03:45.208575+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:31.136135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1e","version":"176'811","reported_seq":3831,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208220+0000","last_change":"2024-09-22T01:03:41.885281+0000","last_active":"2024-09-22T01:03:45.208220+0000","last_peered":"2024-09-22T01:03:45.208220+0000","last_clean":"2024-09-22T01:03:45.208220+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:03:45.208220+0000","last_undegraded":"2024-09-22T01:03:45.208220+0000","last_fullsized":"2024-09-22T01:03:45.208220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:49.748988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1d","version":"175'502","reported_seq":5322,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914876+0000","last_change":"2024-09-22T01:03:41.907575+0000","last_active":"2024-09-22T01:03:45.914876+0000","last_peered":"2024-09-22T01:03:45.914876+0000","last_clean":"2024-09-22T01:03:45.914876+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:03:45.914876+0000","last_undegraded":"2024-09-22T01:03:45.914876+0000","last_fullsized":"2024-09-22T01:03:45.914876+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:35:17.317230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1c","version":"176'1126","reported_seq":10706,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914552+0000","last_change":"2024-09-22T01:03:41.905727+0000","last_active":"2024-09-22T01:03:45.914552+0000","last_peered":"2024-09-22T01:03:45.914552+0000","last_clean":"2024-09-22T01:03:45.914552+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:03:45.914552+0000","last_undegraded":"2024-09-22T01:03:45.914552+0000","last_fullsized":"2024-09-22T01:03:45.914552+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:36.367987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1b","version":"175'1098","reported_seq":3104,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208470+0000","last_change":"2024-09-22T01:03:41.889650+0000","last_active":"2024-09-22T01:03:45.208470+0000","last_peered":"2024-09-22T01:03:45.208470+0000","last_clean":"2024-09-22T01:03:45.208470+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:03:45.208470+0000","last_undegraded":"2024-09-22T01:03:45.208470+0000","last_fullsized":"2024-09-22T01:03:45.208470+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:09.100865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1a","version":"175'484","reported_seq":12360,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898472+0000","last_change":"2024-09-22T01:03:41.912630+0000","last_active":"2024-09-22T01:03:44.898472+0000","last_peered":"2024-09-22T01:03:44.898472+0000","last_clean":"2024-09-22T01:03:44.898472+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:03:44.898472+0000","last_undegraded":"2024-09-22T01:03:44.898472+0000","last_fullsized":"2024-09-22T01:03:44.898472+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:30:44.630235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.19","version":"176'401","reported_seq":10051,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898276+0000","last_change":"2024-09-22T01:03:41.888321+0000","last_active":"2024-09-22T01:03:44.898276+0000","last_peered":"2024-09-22T01:03:44.898276+0000","last_clean":"2024-09-22T01:03:44.898276+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:03:44.898276+0000","last_undegraded":"2024-09-22T01:03:44.898276+0000","last_fullsized":"2024-09-22T01:03:44.898276+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:18.365366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.18","version":"176'793","reported_seq":6603,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.207609+0000","last_change":"2024-09-22T01:03:41.882274+0000","last_active":"2024-09-22T01:03:45.207609+0000","last_peered":"2024-09-22T01:03:45.207609+0000","last_clean":"2024-09-22T01:03:45.207609+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:03:45.207609+0000","last_undegraded":"2024-09-22T01:03:45.207609+0000","last_fullsized":"2024-09-22T01:03:45.207609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:12.860843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.17","version":"175'563","reported_seq":4371,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210578+0000","last_change":"2024-09-22T01:03:41.894428+0000","last_active":"2024-09-22T01:03:45.210578+0000","last_peered":"2024-09-22T01:03:45.210578+0000","last_clean":"2024-09-22T01:03:45.210578+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:03:45.210578+0000","last_undegraded":"2024-09-22T01:03:45.210578+0000","last_fullsized":"2024-09-22T01:03:45.210578+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:55.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.16","version":"175'369","reported_seq":2246,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914752+0000","last_change":"2024-09-22T01:03:41.907362+0000","last_active":"2024-09-22T01:03:45.914752+0000","last_peered":"2024-09-22T01:03:45.914752+0000","last_clean":"2024-09-22T01:03:45.914752+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:03:45.914752+0000","last_undegraded":"2024-09-22T01:03:45.914752+0000","last_fullsized":"2024-09-22T01:03:45.914752+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:08.940607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.15","version":"176'687","reported_seq":6846,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.209418+0000","last_change":"2024-09-22T01:03:41.999115+0000","last_active":"2024-09-22T01:03:45.209418+0000","last_peered":"2024-09-22T01:03:45.209418+0000","last_clean":"2024-09-22T01:03:45.209418+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:03:45.209418+0000","last_undegraded":"2024-09-22T01:03:45.209418+0000","last_fullsized":"2024-09-22T01:03:45.209418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:33.921063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.14","version":"176'563","reported_seq":13946,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898883+0000","last_change":"2024-09-22T01:03:41.912104+0000","last_active":"2024-09-22T01:03:44.898883+0000","last_peered":"2024-09-22T01:03:44.898883+0000","last_clean":"2024-09-22T01:03:44.898883+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:03:44.898883+0000","last_undegraded":"2024-09-22T01:03:44.898883+0000","last_fullsized":"2024-09-22T01:03:44.898883+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:42.625118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":336,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210652+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:03:45.210652+0000","last_peered":"2024-09-22T01:03:45.210652+0000","last_clean":"2024-09-22T01:03:45.210652+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:03:45.210652+0000","last_undegraded":"2024-09-22T01:03:45.210652+0000","last_fullsized":"2024-09-22T01:03:45.210652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"176'605","reported_seq":3282,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208472+0000","last_change":"2024-09-22T01:03:41.887491+0000","last_active":"2024-09-22T01:03:45.208472+0000","last_peered":"2024-09-22T01:03:45.208472+0000","last_clean":"2024-09-22T01:03:45.208472+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:03:45.208472+0000","last_undegraded":"2024-09-22T01:03:45.208472+0000","last_fullsized":"2024-09-22T01:03:45.208472+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:31.037935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898529+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:03:44.898529+0000","last_peered":"2024-09-22T01:03:44.898529+0000","last_clean":"2024-09-22T01:03:44.898529+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:03:44.898529+0000","last_undegraded":"2024-09-22T01:03:44.898529+0000","last_fullsized":"2024-09-22T01:03:44.898529+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'565","reported_seq":3615,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208332+0000","last_change":"2024-09-22T01:03:41.886088+0000","last_active":"2024-09-22T01:03:45.208332+0000","last_peered":"2024-09-22T01:03:45.208332+0000","last_clean":"2024-09-22T01:03:45.208332+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:03:45.208332+0000","last_undegraded":"2024-09-22T01:03:45.208332+0000","last_fullsized":"2024-09-22T01:03:45.208332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:49.543414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":336,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210051+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:03:45.210051+0000","last_peered":"2024-09-22T01:03:45.210051+0000","last_clean":"2024-09-22T01:03:45.210051+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:03:45.210051+0000","last_undegraded":"2024-09-22T01:03:45.210051+0000","last_fullsized":"2024-09-22T01:03:45.210051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'444","reported_seq":4478,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898206+0000","last_change":"2024-09-22T01:03:41.883039+0000","last_active":"2024-09-22T01:03:44.898206+0000","last_peered":"2024-09-22T01:03:44.898206+0000","last_clean":"2024-09-22T01:03:44.898206+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:03:44.898206+0000","last_undegraded":"2024-09-22T01:03:44.898206+0000","last_fullsized":"2024-09-22T01:03:44.898206+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:52.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898178+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:03:44.898178+0000","last_peered":"2024-09-22T01:03:44.898178+0000","last_clean":"2024-09-22T01:03:44.898178+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:03:44.898178+0000","last_undegraded":"2024-09-22T01:03:44.898178+0000","last_fullsized":"2024-09-22T01:03:44.898178+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'680","reported_seq":4649,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.915163+0000","last_change":"2024-09-22T01:03:41.906860+0000","last_active":"2024-09-22T01:03:45.915163+0000","last_peered":"2024-09-22T01:03:45.915163+0000","last_clean":"2024-09-22T01:03:45.915163+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:03:45.915163+0000","last_undegraded":"2024-09-22T01:03:45.915163+0000","last_fullsized":"2024-09-22T01:03:45.915163+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:43.965472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.3","version":"176'736","reported_seq":3057,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898748+0000","last_change":"2024-09-22T01:03:41.913492+0000","last_active":"2024-09-22T01:03:44.898748+0000","last_peered":"2024-09-22T01:03:44.898748+0000","last_clean":"2024-09-22T01:03:44.898748+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:03:44.898748+0000","last_undegraded":"2024-09-22T01:03:44.898748+0000","last_fullsized":"2024-09-22T01:03:44.898748+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:27.079591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.2","version":"21'2","reported_seq":311,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914485+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:03:45.914485+0000","last_peered":"2024-09-22T01:03:45.914485+0000","last_clean":"2024-09-22T01:03:45.914485+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:03:45.914485+0000","last_undegraded":"2024-09-22T01:03:45.914485+0000","last_fullsized":"2024-09-22T01:03:45.914485+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'1653","reported_seq":3909,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898087+0000","last_change":"2024-09-22T01:03:41.883040+0000","last_active":"2024-09-22T01:03:44.898087+0000","last_peered":"2024-09-22T01:03:44.898087+0000","last_clean":"2024-09-22T01:03:44.898087+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:03:44.898087+0000","last_undegraded":"2024-09-22T01:03:44.898087+0000","last_fullsized":"2024-09-22T01:03:44.898087+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1653,"log_dups_size":0,"ondisk_log_size":1653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:43.607282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":336,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.897847+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:03:44.897847+0000","last_peered":"2024-09-22T01:03:44.897847+0000","last_clean":"2024-09-22T01:03:44.897847+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:03:44.897847+0000","last_undegraded":"2024-09-22T01:03:44.897847+0000","last_fullsized":"2024-09-22T01:03:44.897847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'388","reported_seq":3128,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.207994+0000","last_change":"2024-09-22T01:03:41.886773+0000","last_active":"2024-09-22T01:03:45.207994+0000","last_peered":"2024-09-22T01:03:45.207994+0000","last_clean":"2024-09-22T01:03:45.207994+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:03:45.207994+0000","last_undegraded":"2024-09-22T01:03:45.207994+0000","last_fullsized":"2024-09-22T01:03:45.207994+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:39.279522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":351,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208871+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:03:45.208871+0000","last_peered":"2024-09-22T01:03:45.208871+0000","last_clean":"2024-09-22T01:03:45.208871+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:03:45.208871+0000","last_undegraded":"2024-09-22T01:03:45.208871+0000","last_fullsized":"2024-09-22T01:03:45.208871+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'392","reported_seq":2363,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.207870+0000","last_change":"2024-09-22T01:03:41.882465+0000","last_active":"2024-09-22T01:03:45.207870+0000","last_peered":"2024-09-22T01:03:45.207870+0000","last_clean":"2024-09-22T01:03:45.207870+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:03:45.207870+0000","last_undegraded":"2024-09-22T01:03:45.207870+0000","last_fullsized":"2024-09-22T01:03:45.207870+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:12.779986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.3","version":"19'1","reported_seq":292,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914505+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:03:45.914505+0000","last_peered":"2024-09-22T01:03:45.914505+0000","last_clean":"2024-09-22T01:03:45.914505+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:03:45.914505+0000","last_undegraded":"2024-09-22T01:03:45.914505+0000","last_fullsized":"2024-09-22T01:03:45.914505+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":487,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208996+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:03:45.208996+0000","last_peered":"2024-09-22T01:03:45.208996+0000","last_clean":"2024-09-22T01:03:45.208996+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:03:45.208996+0000","last_undegraded":"2024-09-22T01:03:45.208996+0000","last_fullsized":"2024-09-22T01:03:45.208996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'487","reported_seq":10756,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.207692+0000","last_change":"2024-09-22T01:03:41.883318+0000","last_active":"2024-09-22T01:03:45.207692+0000","last_peered":"2024-09-22T01:03:45.207692+0000","last_clean":"2024-09-22T01:03:45.207692+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:03:45.207692+0000","last_undegraded":"2024-09-22T01:03:45.207692+0000","last_fullsized":"2024-09-22T01:03:45.207692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:37.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.9","version":"175'603","reported_seq":2550,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:44.898657+0000","last_change":"2024-09-22T01:03:41.912974+0000","last_active":"2024-09-22T01:03:44.898657+0000","last_peered":"2024-09-22T01:03:44.898657+0000","last_clean":"2024-09-22T01:03:44.898657+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:03:44.898657+0000","last_undegraded":"2024-09-22T01:03:44.898657+0000","last_fullsized":"2024-09-22T01:03:44.898657+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:19.230227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.a","version":"176'801","reported_seq":16424,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.209712+0000","last_change":"2024-09-22T01:03:41.893384+0000","last_active":"2024-09-22T01:03:45.209712+0000","last_peered":"2024-09-22T01:03:45.209712+0000","last_clean":"2024-09-22T01:03:45.209712+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:03:45.209712+0000","last_undegraded":"2024-09-22T01:03:45.209712+0000","last_fullsized":"2024-09-22T01:03:45.209712+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:18.880496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.b","version":"175'701","reported_seq":2951,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208114+0000","last_change":"2024-09-22T01:03:41.889194+0000","last_active":"2024-09-22T01:03:45.208114+0000","last_peered":"2024-09-22T01:03:45.208114+0000","last_clean":"2024-09-22T01:03:45.208114+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:03:45.208114+0000","last_undegraded":"2024-09-22T01:03:45.208114+0000","last_fullsized":"2024-09-22T01:03:45.208114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:52.453227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.c","version":"176'418","reported_seq":1553,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.914616+0000","last_change":"2024-09-22T01:03:41.906335+0000","last_active":"2024-09-22T01:03:45.914616+0000","last_peered":"2024-09-22T01:03:45.914616+0000","last_clean":"2024-09-22T01:03:45.914616+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:03:45.914616+0000","last_undegraded":"2024-09-22T01:03:45.914616+0000","last_fullsized":"2024-09-22T01:03:45.914616+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:26.787651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.d","version":"175'759","reported_seq":5987,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.209765+0000","last_change":"2024-09-22T01:03:41.999171+0000","last_active":"2024-09-22T01:03:45.209765+0000","last_peered":"2024-09-22T01:03:45.209765+0000","last_clean":"2024-09-22T01:03:45.209765+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:03:45.209765+0000","last_undegraded":"2024-09-22T01:03:45.209765+0000","last_fullsized":"2024-09-22T01:03:45.209765+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:25.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.e","version":"176'486","reported_seq":5181,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210341+0000","last_change":"2024-09-22T01:03:42.000884+0000","last_active":"2024-09-22T01:03:45.210341+0000","last_peered":"2024-09-22T01:03:45.210341+0000","last_clean":"2024-09-22T01:03:45.210341+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:03:45.210341+0000","last_undegraded":"2024-09-22T01:03:45.210341+0000","last_fullsized":"2024-09-22T01:03:45.210341+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:01.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.f","version":"174'399","reported_seq":5196,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210682+0000","last_change":"2024-09-22T01:03:42.001967+0000","last_active":"2024-09-22T01:03:45.210682+0000","last_peered":"2024-09-22T01:03:45.210682+0000","last_clean":"2024-09-22T01:03:45.210682+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:03:45.210682+0000","last_undegraded":"2024-09-22T01:03:45.210682+0000","last_fullsized":"2024-09-22T01:03:45.210682+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:14.538603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.10","version":"176'462","reported_seq":3372,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.209612+0000","last_change":"2024-09-22T01:03:41.893276+0000","last_active":"2024-09-22T01:03:45.209612+0000","last_peered":"2024-09-22T01:03:45.209612+0000","last_clean":"2024-09-22T01:03:45.209612+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:03:45.209612+0000","last_undegraded":"2024-09-22T01:03:45.209612+0000","last_fullsized":"2024-09-22T01:03:45.209612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:32.260241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.11","version":"175'680","reported_seq":3027,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.210054+0000","last_change":"2024-09-22T01:03:41.997899+0000","last_active":"2024-09-22T01:03:45.210054+0000","last_peered":"2024-09-22T01:03:45.210054+0000","last_clean":"2024-09-22T01:03:45.210054+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:03:45.210054+0000","last_undegraded":"2024-09-22T01:03:45.210054+0000","last_fullsized":"2024-09-22T01:03:45.210054+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.873636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.12","version":"174'445","reported_seq":3232,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208585+0000","last_change":"2024-09-22T01:03:41.888133+0000","last_active":"2024-09-22T01:03:45.208585+0000","last_peered":"2024-09-22T01:03:45.208585+0000","last_clean":"2024-09-22T01:03:45.208585+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:03:45.208585+0000","last_undegraded":"2024-09-22T01:03:45.208585+0000","last_fullsized":"2024-09-22T01:03:45.208585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:14.342554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.13","version":"175'314","reported_seq":6862,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-22T01:03:45.208000+0000","last_change":"2024-09-22T01:03:41.881304+0000","last_active":"2024-09-22T01:03:45.208000+0000","last_peered":"2024-09-22T01:03:45.208000+0000","last_clean":"2024-09-22T01:03:45.208000+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:03:45.208000+0000","last_undegraded":"2024-09-22T01:03:45.208000+0000","last_fullsized":"2024-09-22T01:03:45.208000+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.277966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19947,"ondisk_log_size":19947,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2150400,"data_stored":1039927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299421,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30236,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341604,"statfs":{"total":96636764160,"available":96605802496,"internally_reserved":0,"allocated":2166784,"data_stored":1045871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60240,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311600,"statfs":{"total":96636764160,"available":96575078400,"internally_reserved":0,"allocated":2744320,"data_stored":1630295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2142208,"data_stored":1034021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47172,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324668,"statfs":{"total":96636764160,"available":96588460032,"internally_reserved":0,"allocated":2138112,"data_stored":1034002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51328,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320512,"statfs":{"total":96636764160,"available":96584204288,"internally_reserved":0,"allocated":2138112,"data_stored":1034002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51344,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320496,"statfs":{"total":96636764160,"available":96584187904,"internally_reserved":0,"allocated":2158592,"data_stored":1039965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47760,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324080,"statfs":{"total":96636764160,"available":96587857920,"internally_reserved":0,"allocated":2744320,"data_stored":1630295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:03:49.625 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T01:03:49.626 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:03:49.626 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-22T01:03:49.626 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:03:49.626 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:03:49.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-22T01:03:49.758 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-22T01:03:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "" 2024-09-22T01:03:49.759 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-22T01:03:54.773 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-22T01:03:54.926 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a: {} 2024-09-22T01:03:54.926 INFO:teuthology.orchestra.run.smithi002.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:03:54.939 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b: {} 2024-09-22T01:03:54.939 INFO:teuthology.orchestra.run.smithi002.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:03:54.949 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c: {} 2024-09-22T01:03:54.949 INFO:teuthology.orchestra.run.smithi002.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:03:54.966 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T01:03:55.761 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T01:03:55.787 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-22T01:03:56.766 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-22T01:03:56.792 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T01:03:57.782 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T01:03:57.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-22T01:03:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-22T01:03:58.816 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T01:03:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T01:03:59.831 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-22T01:04:00.820 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-22T01:04:00.843 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T01:04:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T01:04:01.847 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-22T01:04:02.827 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-22T01:04:02.855 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-22T01:04:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.4 to 1 (10000) 2024-09-22T01:04:03.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-22T01:04:04.856 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-22T01:04:04.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-22T01:04:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.6 to 1 (10000) 2024-09-22T01:04:05.888 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-22T01:04:06.872 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-22T01:04:06.898 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-22T01:04:07.883 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.7 to 1 (10000) 2024-09-22T01:04:07.909 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-22T01:04:08.899 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-22T01:04:08.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-22T01:04:09.904 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.5 to 1 (10000) 2024-09-22T01:04:09.930 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-22T01:04:10.913 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-22T01:04:10.939 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-22T01:04:10.939 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-22T01:04:10.939 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-22T01:04:10.940 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-22T01:04:10.940 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T01:04:10.940 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:04:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:11.250 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":200,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:04:10.911226+0000","last_up_change":"2024-09-22T01:01:26.960327+0000","last_in_change":"2024-09-22T01:03:05.961739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:00:30.699559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:00:36.410605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-22T01:00:46.342370+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":50,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6825","nonce":2277223948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6827","nonce":2277223948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6831","nonce":2277223948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6829","nonce":2277223948}]},"public_addr":"172.21.15.2:6825/2277223948","cluster_addr":"172.21.15.2:6827/2277223948","heartbeat_back_addr":"172.21.15.2:6831/2277223948","heartbeat_front_addr":"172.21.15.2:6829/2277223948","state":["exists","up"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6817","nonce":463340194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6819","nonce":463340194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6823","nonce":463340194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6821","nonce":463340194}]},"public_addr":"172.21.15.2:6817/463340194","cluster_addr":"172.21.15.2:6819/463340194","heartbeat_back_addr":"172.21.15.2:6823/463340194","heartbeat_front_addr":"172.21.15.2:6821/463340194","state":["exists","up"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6801","nonce":387643397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6803","nonce":387643397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6807","nonce":387643397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6805","nonce":387643397}]},"public_addr":"172.21.15.2:6801/387643397","cluster_addr":"172.21.15.2:6803/387643397","heartbeat_back_addr":"172.21.15.2:6807/387643397","heartbeat_front_addr":"172.21.15.2:6805/387643397","state":["exists","up"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6809","nonce":1215210764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6811","nonce":1215210764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6815","nonce":1215210764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6813","nonce":1215210764}]},"public_addr":"172.21.15.2:6809/1215210764","cluster_addr":"172.21.15.2:6811/1215210764","heartbeat_back_addr":"172.21.15.2:6815/1215210764","heartbeat_front_addr":"172.21.15.2:6813/1215210764","state":["exists","up"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6801","nonce":2463696152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6803","nonce":2463696152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6807","nonce":2463696152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6805","nonce":2463696152}]},"public_addr":"172.21.15.119:6801/2463696152","cluster_addr":"172.21.15.119:6803/2463696152","heartbeat_back_addr":"172.21.15.119:6807/2463696152","heartbeat_front_addr":"172.21.15.119:6805/2463696152","state":["exists","up"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":25,"up_from":52,"up_thru":145,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6825","nonce":1737475131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6827","nonce":1737475131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6831","nonce":1737475131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6829","nonce":1737475131}]},"public_addr":"172.21.15.119:6825/1737475131","cluster_addr":"172.21.15.119:6827/1737475131","heartbeat_back_addr":"172.21.15.119:6831/1737475131","heartbeat_front_addr":"172.21.15.119:6829/1737475131","state":["exists","up"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6817","nonce":4220217922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6819","nonce":4220217922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6823","nonce":4220217922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6821","nonce":4220217922}]},"public_addr":"172.21.15.119:6817/4220217922","cluster_addr":"172.21.15.119:6819/4220217922","heartbeat_back_addr":"172.21.15.119:6823/4220217922","heartbeat_front_addr":"172.21.15.119:6821/4220217922","state":["exists","up"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6809","nonce":3867156106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6811","nonce":3867156106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6815","nonce":3867156106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6813","nonce":3867156106}]},"public_addr":"172.21.15.119:6809/3867156106","cluster_addr":"172.21.15.119:6811/3867156106","heartbeat_back_addr":"172.21.15.119:6815/3867156106","heartbeat_front_addr":"172.21.15.119:6813/3867156106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.556128+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.159753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.766199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.155529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.226232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.243346+0000","dead_epoch":26},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.143084+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.524369+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.17","osds":[6,1]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":1,"to":2}]},{"pgid":"3.a","mappings":[{"from":4,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:04:11.271 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T01:04:11.271 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:04:11.580 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:11.580 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":200,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:04:10.911226+0000","last_up_change":"2024-09-22T01:01:26.960327+0000","last_in_change":"2024-09-22T01:03:05.961739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:00:30.699559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:00:36.410605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-22T01:00:46.342370+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":50,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6825","nonce":2277223948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6827","nonce":2277223948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6831","nonce":2277223948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6829","nonce":2277223948}]},"public_addr":"172.21.15.2:6825/2277223948","cluster_addr":"172.21.15.2:6827/2277223948","heartbeat_back_addr":"172.21.15.2:6831/2277223948","heartbeat_front_addr":"172.21.15.2:6829/2277223948","state":["exists","up"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6817","nonce":463340194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6819","nonce":463340194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6823","nonce":463340194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6821","nonce":463340194}]},"public_addr":"172.21.15.2:6817/463340194","cluster_addr":"172.21.15.2:6819/463340194","heartbeat_back_addr":"172.21.15.2:6823/463340194","heartbeat_front_addr":"172.21.15.2:6821/463340194","state":["exists","up"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6801","nonce":387643397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6803","nonce":387643397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6807","nonce":387643397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6805","nonce":387643397}]},"public_addr":"172.21.15.2:6801/387643397","cluster_addr":"172.21.15.2:6803/387643397","heartbeat_back_addr":"172.21.15.2:6807/387643397","heartbeat_front_addr":"172.21.15.2:6805/387643397","state":["exists","up"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6809","nonce":1215210764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6811","nonce":1215210764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6815","nonce":1215210764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6813","nonce":1215210764}]},"public_addr":"172.21.15.2:6809/1215210764","cluster_addr":"172.21.15.2:6811/1215210764","heartbeat_back_addr":"172.21.15.2:6815/1215210764","heartbeat_front_addr":"172.21.15.2:6813/1215210764","state":["exists","up"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6801","nonce":2463696152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6803","nonce":2463696152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6807","nonce":2463696152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6805","nonce":2463696152}]},"public_addr":"172.21.15.119:6801/2463696152","cluster_addr":"172.21.15.119:6803/2463696152","heartbeat_back_addr":"172.21.15.119:6807/2463696152","heartbeat_front_addr":"172.21.15.119:6805/2463696152","state":["exists","up"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":25,"up_from":52,"up_thru":145,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6825","nonce":1737475131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6827","nonce":1737475131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6831","nonce":1737475131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6829","nonce":1737475131}]},"public_addr":"172.21.15.119:6825/1737475131","cluster_addr":"172.21.15.119:6827/1737475131","heartbeat_back_addr":"172.21.15.119:6831/1737475131","heartbeat_front_addr":"172.21.15.119:6829/1737475131","state":["exists","up"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6817","nonce":4220217922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6819","nonce":4220217922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6823","nonce":4220217922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6821","nonce":4220217922}]},"public_addr":"172.21.15.119:6817/4220217922","cluster_addr":"172.21.15.119:6819/4220217922","heartbeat_back_addr":"172.21.15.119:6823/4220217922","heartbeat_front_addr":"172.21.15.119:6821/4220217922","state":["exists","up"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6809","nonce":3867156106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6811","nonce":3867156106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6815","nonce":3867156106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6813","nonce":3867156106}]},"public_addr":"172.21.15.119:6809/3867156106","cluster_addr":"172.21.15.119:6811/3867156106","heartbeat_back_addr":"172.21.15.119:6815/3867156106","heartbeat_front_addr":"172.21.15.119:6813/3867156106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.556128+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.159753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.766199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.155529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.226232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.243346+0000","dead_epoch":26},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.143084+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.524369+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.17","osds":[6,1]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":1,"to":2}]},{"pgid":"3.a","mappings":[{"from":4,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:04:11.600 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T01:04:11.600 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T01:04:11.601 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T01:04:11.601 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T01:04:11.601 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-22T01:04:11.602 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-22T01:04:11.602 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-22T01:04:11.603 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-22T01:04:11.864 INFO:teuthology.orchestra.run.smithi002.stdout:60129542192 2024-09-22T01:04:12.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:04:12.204 INFO:teuthology.orchestra.run.smithi002.stdout:60129542193 2024-09-22T01:04:12.204 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-22T01:04:12.205 INFO:teuthology.orchestra.run.smithi002.stdout:60129542194 2024-09-22T01:04:12.205 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T01:04:12.206 INFO:teuthology.orchestra.run.smithi002.stdout:60129542193 2024-09-22T01:04:12.207 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-22T01:04:12.207 INFO:teuthology.orchestra.run.smithi002.stdout:60129542193 2024-09-22T01:04:12.207 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T01:04:12.208 INFO:teuthology.orchestra.run.smithi002.stdout:223338299427 2024-09-22T01:04:12.208 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-22T01:04:12.209 INFO:teuthology.orchestra.run.smithi002.stdout:64424509489 2024-09-22T01:04:12.210 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-22T01:04:12.210 INFO:teuthology.orchestra.run.smithi002.stdout:60129542192 2024-09-22T01:04:12.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T01:04:12.778 INFO:teuthology.orchestra.run.smithi002.stdout:60129542191 2024-09-22T01:04:12.789 INFO:teuthology.orchestra.run.smithi002.stdout:223338299425 2024-09-22T01:04:12.794 INFO:teuthology.orchestra.run.smithi002.stdout:60129542190 2024-09-22T01:04:12.800 INFO:teuthology.orchestra.run.smithi002.stdout:60129542190 2024-09-22T01:04:12.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542191 for osd.7 2024-09-22T01:04:12.811 INFO:teuthology.orchestra.run.smithi002.stdout:60129542191 2024-09-22T01:04:12.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 223338299427 got 223338299425 for osd.5 2024-09-22T01:04:12.825 INFO:teuthology.orchestra.run.smithi002.stdout:60129542192 2024-09-22T01:04:12.826 INFO:teuthology.orchestra.run.smithi002.stdout:64424509487 2024-09-22T01:04:12.828 INFO:teuthology.orchestra.run.smithi002.stdout:60129542191 2024-09-22T01:04:12.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542190 for osd.3 2024-09-22T01:04:12.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542190 for osd.2 2024-09-22T01:04:12.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542191 for osd.0 2024-09-22T01:04:12.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542192 for osd.1 2024-09-22T01:04:12.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509489 got 64424509487 for osd.6 2024-09-22T01:04:12.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542191 for osd.4 2024-09-22T01:04:13.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-22T01:04:13.824 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-22T01:04:13.834 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:04:13.835 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T01:04:13.845 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T01:04:13.852 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T01:04:13.852 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-22T01:04:13.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-22T01:04:14.354 INFO:teuthology.orchestra.run.smithi002.stdout:60129542193 2024-09-22T01:04:14.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542193 for osd.7 2024-09-22T01:04:14.378 DEBUG:teuthology.parallel:result is None 2024-09-22T01:04:14.402 INFO:teuthology.orchestra.run.smithi002.stdout:64424509489 2024-09-22T01:04:14.402 INFO:teuthology.orchestra.run.smithi002.stdout:60129542192 2024-09-22T01:04:14.411 INFO:teuthology.orchestra.run.smithi002.stdout:60129542192 2024-09-22T01:04:14.421 INFO:teuthology.orchestra.run.smithi002.stdout:60129542194 2024-09-22T01:04:14.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509489 got 64424509489 for osd.6 2024-09-22T01:04:14.425 DEBUG:teuthology.parallel:result is None 2024-09-22T01:04:14.431 INFO:teuthology.orchestra.run.smithi002.stdout:60129542193 2024-09-22T01:04:14.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542192 for osd.3 2024-09-22T01:04:14.435 DEBUG:teuthology.parallel:result is None 2024-09-22T01:04:14.438 INFO:teuthology.orchestra.run.smithi002.stdout:223338299427 2024-09-22T01:04:14.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542192 for osd.2 2024-09-22T01:04:14.446 DEBUG:teuthology.parallel:result is None 2024-09-22T01:04:14.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542194 for osd.1 2024-09-22T01:04:14.449 DEBUG:teuthology.parallel:result is None 2024-09-22T01:04:14.456 INFO:teuthology.orchestra.run.smithi002.stdout:60129542193 2024-09-22T01:04:14.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 223338299427 got 223338299427 for osd.5 2024-09-22T01:04:14.460 DEBUG:teuthology.parallel:result is None 2024-09-22T01:04:14.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542193 for osd.0 2024-09-22T01:04:14.462 DEBUG:teuthology.parallel:result is None 2024-09-22T01:04:14.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542193 for osd.4 2024-09-22T01:04:14.475 DEBUG:teuthology.parallel:result is None 2024-09-22T01:04:14.475 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T01:04:14.475 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:04:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:14.799 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:04:14.815 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-22T01:04:14.676383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20067,"ondisk_log_size":20067,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378892,"kb_used_data":19980,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754595828,"statfs":{"total":773094113280,"available":772706127872,"internally_reserved":0,"allocated":20459520,"data_stored":10490563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001700"},"pg_stats":[{"pgid":"3.1f","version":"176'532","reported_seq":7206,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313143+0000","last_change":"2024-09-22T01:03:41.888720+0000","last_active":"2024-09-22T01:04:11.313143+0000","last_peered":"2024-09-22T01:04:11.313143+0000","last_clean":"2024-09-22T01:04:11.313143+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:04:11.313143+0000","last_undegraded":"2024-09-22T01:04:11.313143+0000","last_fullsized":"2024-09-22T01:04:11.313143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:31.136135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"176'811","reported_seq":3869,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312644+0000","last_change":"2024-09-22T01:03:41.885281+0000","last_active":"2024-09-22T01:04:11.312644+0000","last_peered":"2024-09-22T01:04:11.312644+0000","last_clean":"2024-09-22T01:04:11.312644+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:04:11.312644+0000","last_undegraded":"2024-09-22T01:04:11.312644+0000","last_fullsized":"2024-09-22T01:04:11.312644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:49.748988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'502","reported_seq":5361,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313981+0000","last_change":"2024-09-22T01:03:41.907575+0000","last_active":"2024-09-22T01:04:11.313981+0000","last_peered":"2024-09-22T01:04:11.313981+0000","last_clean":"2024-09-22T01:04:11.313981+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:04:11.313981+0000","last_undegraded":"2024-09-22T01:04:11.313981+0000","last_fullsized":"2024-09-22T01:04:11.313981+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:35:17.317230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1126","reported_seq":10745,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314011+0000","last_change":"2024-09-22T01:03:41.905727+0000","last_active":"2024-09-22T01:04:11.314011+0000","last_peered":"2024-09-22T01:04:11.314011+0000","last_clean":"2024-09-22T01:04:11.314011+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:04:11.314011+0000","last_undegraded":"2024-09-22T01:04:11.314011+0000","last_fullsized":"2024-09-22T01:04:11.314011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:36.367987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'1098","reported_seq":3143,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313103+0000","last_change":"2024-09-22T01:03:41.889650+0000","last_active":"2024-09-22T01:04:11.313103+0000","last_peered":"2024-09-22T01:04:11.313103+0000","last_clean":"2024-09-22T01:04:11.313103+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:04:11.313103+0000","last_undegraded":"2024-09-22T01:04:11.313103+0000","last_fullsized":"2024-09-22T01:04:11.313103+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:09.100865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'484","reported_seq":12400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912630+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:30:44.630235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'401","reported_seq":10093,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919216+0000","last_change":"2024-09-22T01:03:41.888321+0000","last_active":"2024-09-22T01:04:10.919216+0000","last_peered":"2024-09-22T01:04:10.919216+0000","last_clean":"2024-09-22T01:04:10.919216+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:04:10.919216+0000","last_undegraded":"2024-09-22T01:04:10.919216+0000","last_fullsized":"2024-09-22T01:04:10.919216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:18.365366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"176'793","reported_seq":6641,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312433+0000","last_change":"2024-09-22T01:03:41.882274+0000","last_active":"2024-09-22T01:04:11.312433+0000","last_peered":"2024-09-22T01:04:11.312433+0000","last_clean":"2024-09-22T01:04:11.312433+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:04:11.312433+0000","last_undegraded":"2024-09-22T01:04:11.312433+0000","last_fullsized":"2024-09-22T01:04:11.312433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:12.860843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'563","reported_seq":4411,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315369+0000","last_change":"2024-09-22T01:03:41.894428+0000","last_active":"2024-09-22T01:04:11.315369+0000","last_peered":"2024-09-22T01:04:11.315369+0000","last_clean":"2024-09-22T01:04:11.315369+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:04:11.315369+0000","last_undegraded":"2024-09-22T01:04:11.315369+0000","last_fullsized":"2024-09-22T01:04:11.315369+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:55.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"175'369","reported_seq":2285,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314632+0000","last_change":"2024-09-22T01:03:41.907362+0000","last_active":"2024-09-22T01:04:11.314632+0000","last_peered":"2024-09-22T01:04:11.314632+0000","last_clean":"2024-09-22T01:04:11.314632+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:04:11.314632+0000","last_undegraded":"2024-09-22T01:04:11.314632+0000","last_fullsized":"2024-09-22T01:04:11.314632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:08.940607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'687","reported_seq":6885,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313654+0000","last_change":"2024-09-22T01:03:41.999115+0000","last_active":"2024-09-22T01:04:11.313654+0000","last_peered":"2024-09-22T01:04:11.313654+0000","last_clean":"2024-09-22T01:04:11.313654+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:04:11.313654+0000","last_undegraded":"2024-09-22T01:04:11.313654+0000","last_fullsized":"2024-09-22T01:04:11.313654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:33.921063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'563","reported_seq":13986,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918873+0000","last_change":"2024-09-22T01:03:41.912104+0000","last_active":"2024-09-22T01:04:10.918873+0000","last_peered":"2024-09-22T01:04:10.918873+0000","last_clean":"2024-09-22T01:04:10.918873+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:04:10.918873+0000","last_undegraded":"2024-09-22T01:04:10.918873+0000","last_fullsized":"2024-09-22T01:04:10.918873+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:42.625118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":376,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315371+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:04:11.315371+0000","last_peered":"2024-09-22T01:04:11.315371+0000","last_clean":"2024-09-22T01:04:11.315371+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:04:11.315371+0000","last_undegraded":"2024-09-22T01:04:11.315371+0000","last_fullsized":"2024-09-22T01:04:11.315371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"176'605","reported_seq":3321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313152+0000","last_change":"2024-09-22T01:03:41.887491+0000","last_active":"2024-09-22T01:04:11.313152+0000","last_peered":"2024-09-22T01:04:11.313152+0000","last_clean":"2024-09-22T01:04:11.313152+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:04:11.313152+0000","last_undegraded":"2024-09-22T01:04:11.313152+0000","last_fullsized":"2024-09-22T01:04:11.313152+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:31.037935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919602+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:04:10.919602+0000","last_peered":"2024-09-22T01:04:10.919602+0000","last_clean":"2024-09-22T01:04:10.919602+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:04:10.919602+0000","last_undegraded":"2024-09-22T01:04:10.919602+0000","last_fullsized":"2024-09-22T01:04:10.919602+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'565","reported_seq":3653,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312775+0000","last_change":"2024-09-22T01:03:41.886088+0000","last_active":"2024-09-22T01:04:11.312775+0000","last_peered":"2024-09-22T01:04:11.312775+0000","last_clean":"2024-09-22T01:04:11.312775+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:04:11.312775+0000","last_undegraded":"2024-09-22T01:04:11.312775+0000","last_fullsized":"2024-09-22T01:04:11.312775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:49.543414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":375,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313917+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:04:11.313917+0000","last_peered":"2024-09-22T01:04:11.313917+0000","last_clean":"2024-09-22T01:04:11.313917+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:04:11.313917+0000","last_undegraded":"2024-09-22T01:04:11.313917+0000","last_fullsized":"2024-09-22T01:04:11.313917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'444","reported_seq":4520,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919765+0000","last_change":"2024-09-22T01:03:41.883039+0000","last_active":"2024-09-22T01:04:10.919765+0000","last_peered":"2024-09-22T01:04:10.919765+0000","last_clean":"2024-09-22T01:04:10.919765+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:04:10.919765+0000","last_undegraded":"2024-09-22T01:04:10.919765+0000","last_fullsized":"2024-09-22T01:04:10.919765+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:52.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919513+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:04:10.919513+0000","last_peered":"2024-09-22T01:04:10.919513+0000","last_clean":"2024-09-22T01:04:10.919513+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:04:10.919513+0000","last_undegraded":"2024-09-22T01:04:10.919513+0000","last_fullsized":"2024-09-22T01:04:10.919513+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'680","reported_seq":4687,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314274+0000","last_change":"2024-09-22T01:03:41.906860+0000","last_active":"2024-09-22T01:04:11.314274+0000","last_peered":"2024-09-22T01:04:11.314274+0000","last_clean":"2024-09-22T01:04:11.314274+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:04:11.314274+0000","last_undegraded":"2024-09-22T01:04:11.314274+0000","last_fullsized":"2024-09-22T01:04:11.314274+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:43.965472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"176'736","reported_seq":3097,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918646+0000","last_change":"2024-09-22T01:03:41.913492+0000","last_active":"2024-09-22T01:04:10.918646+0000","last_peered":"2024-09-22T01:04:10.918646+0000","last_clean":"2024-09-22T01:04:10.918646+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:04:10.918646+0000","last_undegraded":"2024-09-22T01:04:10.918646+0000","last_fullsized":"2024-09-22T01:04:10.918646+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:27.079591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":349,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314672+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:04:11.314672+0000","last_peered":"2024-09-22T01:04:11.314672+0000","last_clean":"2024-09-22T01:04:11.314672+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:04:11.314672+0000","last_undegraded":"2024-09-22T01:04:11.314672+0000","last_fullsized":"2024-09-22T01:04:11.314672+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'1653","reported_seq":3951,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919226+0000","last_change":"2024-09-22T01:03:41.883040+0000","last_active":"2024-09-22T01:04:10.919226+0000","last_peered":"2024-09-22T01:04:10.919226+0000","last_clean":"2024-09-22T01:04:10.919226+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:04:10.919226+0000","last_undegraded":"2024-09-22T01:04:10.919226+0000","last_fullsized":"2024-09-22T01:04:10.919226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1653,"log_dups_size":0,"ondisk_log_size":1653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:43.607282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":377,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.916415+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:04:10.916415+0000","last_peered":"2024-09-22T01:04:10.916415+0000","last_clean":"2024-09-22T01:04:10.916415+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:04:10.916415+0000","last_undegraded":"2024-09-22T01:04:10.916415+0000","last_fullsized":"2024-09-22T01:04:10.916415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'388","reported_seq":3167,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312876+0000","last_change":"2024-09-22T01:03:41.886773+0000","last_active":"2024-09-22T01:04:11.312876+0000","last_peered":"2024-09-22T01:04:11.312876+0000","last_clean":"2024-09-22T01:04:11.312876+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:04:11.312876+0000","last_undegraded":"2024-09-22T01:04:11.312876+0000","last_fullsized":"2024-09-22T01:04:11.312876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:39.279522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":390,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314477+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:04:11.314477+0000","last_peered":"2024-09-22T01:04:11.314477+0000","last_clean":"2024-09-22T01:04:11.314477+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:04:11.314477+0000","last_undegraded":"2024-09-22T01:04:11.314477+0000","last_fullsized":"2024-09-22T01:04:11.314477+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'392","reported_seq":2401,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312604+0000","last_change":"2024-09-22T01:03:41.882465+0000","last_active":"2024-09-22T01:04:11.312604+0000","last_peered":"2024-09-22T01:04:11.312604+0000","last_clean":"2024-09-22T01:04:11.312604+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:04:11.312604+0000","last_undegraded":"2024-09-22T01:04:11.312604+0000","last_fullsized":"2024-09-22T01:04:11.312604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:12.779986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314378+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:04:11.314378+0000","last_peered":"2024-09-22T01:04:11.314378+0000","last_clean":"2024-09-22T01:04:11.314378+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:04:11.314378+0000","last_undegraded":"2024-09-22T01:04:11.314378+0000","last_fullsized":"2024-09-22T01:04:11.314378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":526,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313661+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:04:11.313661+0000","last_peered":"2024-09-22T01:04:11.313661+0000","last_clean":"2024-09-22T01:04:11.313661+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:04:11.313661+0000","last_undegraded":"2024-09-22T01:04:11.313661+0000","last_fullsized":"2024-09-22T01:04:11.313661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'487","reported_seq":10794,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312452+0000","last_change":"2024-09-22T01:03:41.883318+0000","last_active":"2024-09-22T01:04:11.312452+0000","last_peered":"2024-09-22T01:04:11.312452+0000","last_clean":"2024-09-22T01:04:11.312452+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:04:11.312452+0000","last_undegraded":"2024-09-22T01:04:11.312452+0000","last_fullsized":"2024-09-22T01:04:11.312452+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:37.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'603","reported_seq":2590,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912974+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:19.230227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'801","reported_seq":16464,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315079+0000","last_change":"2024-09-22T01:03:41.893384+0000","last_active":"2024-09-22T01:04:11.315079+0000","last_peered":"2024-09-22T01:04:11.315079+0000","last_clean":"2024-09-22T01:04:11.315079+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:04:11.315079+0000","last_undegraded":"2024-09-22T01:04:11.315079+0000","last_fullsized":"2024-09-22T01:04:11.315079+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:18.880496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"175'701","reported_seq":2989,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312731+0000","last_change":"2024-09-22T01:03:41.889194+0000","last_active":"2024-09-22T01:04:11.312731+0000","last_peered":"2024-09-22T01:04:11.312731+0000","last_clean":"2024-09-22T01:04:11.312731+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:04:11.312731+0000","last_undegraded":"2024-09-22T01:04:11.312731+0000","last_fullsized":"2024-09-22T01:04:11.312731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:52.453227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'418","reported_seq":1591,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314308+0000","last_change":"2024-09-22T01:03:41.906335+0000","last_active":"2024-09-22T01:04:11.314308+0000","last_peered":"2024-09-22T01:04:11.314308+0000","last_clean":"2024-09-22T01:04:11.314308+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:04:11.314308+0000","last_undegraded":"2024-09-22T01:04:11.314308+0000","last_fullsized":"2024-09-22T01:04:11.314308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:26.787651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'759","reported_seq":6026,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313835+0000","last_change":"2024-09-22T01:03:41.999171+0000","last_active":"2024-09-22T01:04:11.313835+0000","last_peered":"2024-09-22T01:04:11.313835+0000","last_clean":"2024-09-22T01:04:11.313835+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:04:11.313835+0000","last_undegraded":"2024-09-22T01:04:11.313835+0000","last_fullsized":"2024-09-22T01:04:11.313835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:25.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'486","reported_seq":5220,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314145+0000","last_change":"2024-09-22T01:03:42.000884+0000","last_active":"2024-09-22T01:04:11.314145+0000","last_peered":"2024-09-22T01:04:11.314145+0000","last_clean":"2024-09-22T01:04:11.314145+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:04:11.314145+0000","last_undegraded":"2024-09-22T01:04:11.314145+0000","last_fullsized":"2024-09-22T01:04:11.314145+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:01.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"174'399","reported_seq":5235,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313998+0000","last_change":"2024-09-22T01:03:42.001967+0000","last_active":"2024-09-22T01:04:11.313998+0000","last_peered":"2024-09-22T01:04:11.313998+0000","last_clean":"2024-09-22T01:04:11.313998+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:04:11.313998+0000","last_undegraded":"2024-09-22T01:04:11.313998+0000","last_fullsized":"2024-09-22T01:04:11.313998+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:14.538603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'462","reported_seq":3412,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315076+0000","last_change":"2024-09-22T01:03:41.893276+0000","last_active":"2024-09-22T01:04:11.315076+0000","last_peered":"2024-09-22T01:04:11.315076+0000","last_clean":"2024-09-22T01:04:11.315076+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:04:11.315076+0000","last_undegraded":"2024-09-22T01:04:11.315076+0000","last_fullsized":"2024-09-22T01:04:11.315076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:32.260241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'680","reported_seq":3066,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313567+0000","last_change":"2024-09-22T01:03:41.997899+0000","last_active":"2024-09-22T01:04:11.313567+0000","last_peered":"2024-09-22T01:04:11.313567+0000","last_clean":"2024-09-22T01:04:11.313567+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:04:11.313567+0000","last_undegraded":"2024-09-22T01:04:11.313567+0000","last_fullsized":"2024-09-22T01:04:11.313567+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.873636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'445","reported_seq":3271,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312929+0000","last_change":"2024-09-22T01:03:41.888133+0000","last_active":"2024-09-22T01:04:11.312929+0000","last_peered":"2024-09-22T01:04:11.312929+0000","last_clean":"2024-09-22T01:04:11.312929+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:04:11.312929+0000","last_undegraded":"2024-09-22T01:04:11.312929+0000","last_fullsized":"2024-09-22T01:04:11.312929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:14.342554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'314","reported_seq":6900,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312720+0000","last_change":"2024-09-22T01:03:41.881304+0000","last_active":"2024-09-22T01:04:11.312720+0000","last_peered":"2024-09-22T01:04:11.312720+0000","last_clean":"2024-09-22T01:04:11.312720+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:04:11.312720+0000","last_undegraded":"2024-09-22T01:04:11.312720+0000","last_fullsized":"2024-09-22T01:04:11.312720+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.277966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19947,"ondisk_log_size":19947,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33548,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338292,"statfs":{"total":96636764160,"available":96602411008,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299427,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":2344,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":2400256,"data_stored":1158550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55752,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316088,"statfs":{"total":96636764160,"available":96579674112,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47432,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324408,"statfs":{"total":96636764160,"available":96588193792,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51604,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320236,"statfs":{"total":96636764160,"available":96583921664,"internally_reserved":0,"allocated":2416640,"data_stored":1164498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48008,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323832,"statfs":{"total":96636764160,"available":96587603968,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:04:14.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:04:15.110 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:15.110 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:04:15.126 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-22T01:04:14.676383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20067,"ondisk_log_size":20067,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378892,"kb_used_data":19980,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754595828,"statfs":{"total":773094113280,"available":772706127872,"internally_reserved":0,"allocated":20459520,"data_stored":10490563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001700"},"pg_stats":[{"pgid":"3.1f","version":"176'532","reported_seq":7206,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313143+0000","last_change":"2024-09-22T01:03:41.888720+0000","last_active":"2024-09-22T01:04:11.313143+0000","last_peered":"2024-09-22T01:04:11.313143+0000","last_clean":"2024-09-22T01:04:11.313143+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:04:11.313143+0000","last_undegraded":"2024-09-22T01:04:11.313143+0000","last_fullsized":"2024-09-22T01:04:11.313143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:31.136135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"176'811","reported_seq":3869,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312644+0000","last_change":"2024-09-22T01:03:41.885281+0000","last_active":"2024-09-22T01:04:11.312644+0000","last_peered":"2024-09-22T01:04:11.312644+0000","last_clean":"2024-09-22T01:04:11.312644+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:04:11.312644+0000","last_undegraded":"2024-09-22T01:04:11.312644+0000","last_fullsized":"2024-09-22T01:04:11.312644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:49.748988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'502","reported_seq":5361,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313981+0000","last_change":"2024-09-22T01:03:41.907575+0000","last_active":"2024-09-22T01:04:11.313981+0000","last_peered":"2024-09-22T01:04:11.313981+0000","last_clean":"2024-09-22T01:04:11.313981+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:04:11.313981+0000","last_undegraded":"2024-09-22T01:04:11.313981+0000","last_fullsized":"2024-09-22T01:04:11.313981+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:35:17.317230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1126","reported_seq":10745,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314011+0000","last_change":"2024-09-22T01:03:41.905727+0000","last_active":"2024-09-22T01:04:11.314011+0000","last_peered":"2024-09-22T01:04:11.314011+0000","last_clean":"2024-09-22T01:04:11.314011+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:04:11.314011+0000","last_undegraded":"2024-09-22T01:04:11.314011+0000","last_fullsized":"2024-09-22T01:04:11.314011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:36.367987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'1098","reported_seq":3143,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313103+0000","last_change":"2024-09-22T01:03:41.889650+0000","last_active":"2024-09-22T01:04:11.313103+0000","last_peered":"2024-09-22T01:04:11.313103+0000","last_clean":"2024-09-22T01:04:11.313103+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:04:11.313103+0000","last_undegraded":"2024-09-22T01:04:11.313103+0000","last_fullsized":"2024-09-22T01:04:11.313103+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:09.100865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'484","reported_seq":12400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912630+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:30:44.630235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'401","reported_seq":10093,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919216+0000","last_change":"2024-09-22T01:03:41.888321+0000","last_active":"2024-09-22T01:04:10.919216+0000","last_peered":"2024-09-22T01:04:10.919216+0000","last_clean":"2024-09-22T01:04:10.919216+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:04:10.919216+0000","last_undegraded":"2024-09-22T01:04:10.919216+0000","last_fullsized":"2024-09-22T01:04:10.919216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:18.365366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"176'793","reported_seq":6641,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312433+0000","last_change":"2024-09-22T01:03:41.882274+0000","last_active":"2024-09-22T01:04:11.312433+0000","last_peered":"2024-09-22T01:04:11.312433+0000","last_clean":"2024-09-22T01:04:11.312433+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:04:11.312433+0000","last_undegraded":"2024-09-22T01:04:11.312433+0000","last_fullsized":"2024-09-22T01:04:11.312433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:12.860843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'563","reported_seq":4411,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315369+0000","last_change":"2024-09-22T01:03:41.894428+0000","last_active":"2024-09-22T01:04:11.315369+0000","last_peered":"2024-09-22T01:04:11.315369+0000","last_clean":"2024-09-22T01:04:11.315369+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:04:11.315369+0000","last_undegraded":"2024-09-22T01:04:11.315369+0000","last_fullsized":"2024-09-22T01:04:11.315369+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:55.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"175'369","reported_seq":2285,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314632+0000","last_change":"2024-09-22T01:03:41.907362+0000","last_active":"2024-09-22T01:04:11.314632+0000","last_peered":"2024-09-22T01:04:11.314632+0000","last_clean":"2024-09-22T01:04:11.314632+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:04:11.314632+0000","last_undegraded":"2024-09-22T01:04:11.314632+0000","last_fullsized":"2024-09-22T01:04:11.314632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:08.940607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'687","reported_seq":6885,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313654+0000","last_change":"2024-09-22T01:03:41.999115+0000","last_active":"2024-09-22T01:04:11.313654+0000","last_peered":"2024-09-22T01:04:11.313654+0000","last_clean":"2024-09-22T01:04:11.313654+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:04:11.313654+0000","last_undegraded":"2024-09-22T01:04:11.313654+0000","last_fullsized":"2024-09-22T01:04:11.313654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:33.921063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'563","reported_seq":13986,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918873+0000","last_change":"2024-09-22T01:03:41.912104+0000","last_active":"2024-09-22T01:04:10.918873+0000","last_peered":"2024-09-22T01:04:10.918873+0000","last_clean":"2024-09-22T01:04:10.918873+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:04:10.918873+0000","last_undegraded":"2024-09-22T01:04:10.918873+0000","last_fullsized":"2024-09-22T01:04:10.918873+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:42.625118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":376,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315371+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:04:11.315371+0000","last_peered":"2024-09-22T01:04:11.315371+0000","last_clean":"2024-09-22T01:04:11.315371+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:04:11.315371+0000","last_undegraded":"2024-09-22T01:04:11.315371+0000","last_fullsized":"2024-09-22T01:04:11.315371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"176'605","reported_seq":3321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313152+0000","last_change":"2024-09-22T01:03:41.887491+0000","last_active":"2024-09-22T01:04:11.313152+0000","last_peered":"2024-09-22T01:04:11.313152+0000","last_clean":"2024-09-22T01:04:11.313152+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:04:11.313152+0000","last_undegraded":"2024-09-22T01:04:11.313152+0000","last_fullsized":"2024-09-22T01:04:11.313152+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:31.037935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919602+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:04:10.919602+0000","last_peered":"2024-09-22T01:04:10.919602+0000","last_clean":"2024-09-22T01:04:10.919602+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:04:10.919602+0000","last_undegraded":"2024-09-22T01:04:10.919602+0000","last_fullsized":"2024-09-22T01:04:10.919602+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'565","reported_seq":3653,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312775+0000","last_change":"2024-09-22T01:03:41.886088+0000","last_active":"2024-09-22T01:04:11.312775+0000","last_peered":"2024-09-22T01:04:11.312775+0000","last_clean":"2024-09-22T01:04:11.312775+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:04:11.312775+0000","last_undegraded":"2024-09-22T01:04:11.312775+0000","last_fullsized":"2024-09-22T01:04:11.312775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:49.543414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":375,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313917+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:04:11.313917+0000","last_peered":"2024-09-22T01:04:11.313917+0000","last_clean":"2024-09-22T01:04:11.313917+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:04:11.313917+0000","last_undegraded":"2024-09-22T01:04:11.313917+0000","last_fullsized":"2024-09-22T01:04:11.313917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'444","reported_seq":4520,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919765+0000","last_change":"2024-09-22T01:03:41.883039+0000","last_active":"2024-09-22T01:04:10.919765+0000","last_peered":"2024-09-22T01:04:10.919765+0000","last_clean":"2024-09-22T01:04:10.919765+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:04:10.919765+0000","last_undegraded":"2024-09-22T01:04:10.919765+0000","last_fullsized":"2024-09-22T01:04:10.919765+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:52.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919513+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:04:10.919513+0000","last_peered":"2024-09-22T01:04:10.919513+0000","last_clean":"2024-09-22T01:04:10.919513+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:04:10.919513+0000","last_undegraded":"2024-09-22T01:04:10.919513+0000","last_fullsized":"2024-09-22T01:04:10.919513+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'680","reported_seq":4687,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314274+0000","last_change":"2024-09-22T01:03:41.906860+0000","last_active":"2024-09-22T01:04:11.314274+0000","last_peered":"2024-09-22T01:04:11.314274+0000","last_clean":"2024-09-22T01:04:11.314274+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:04:11.314274+0000","last_undegraded":"2024-09-22T01:04:11.314274+0000","last_fullsized":"2024-09-22T01:04:11.314274+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:43.965472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"176'736","reported_seq":3097,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918646+0000","last_change":"2024-09-22T01:03:41.913492+0000","last_active":"2024-09-22T01:04:10.918646+0000","last_peered":"2024-09-22T01:04:10.918646+0000","last_clean":"2024-09-22T01:04:10.918646+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:04:10.918646+0000","last_undegraded":"2024-09-22T01:04:10.918646+0000","last_fullsized":"2024-09-22T01:04:10.918646+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:27.079591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":349,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314672+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:04:11.314672+0000","last_peered":"2024-09-22T01:04:11.314672+0000","last_clean":"2024-09-22T01:04:11.314672+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:04:11.314672+0000","last_undegraded":"2024-09-22T01:04:11.314672+0000","last_fullsized":"2024-09-22T01:04:11.314672+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'1653","reported_seq":3951,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919226+0000","last_change":"2024-09-22T01:03:41.883040+0000","last_active":"2024-09-22T01:04:10.919226+0000","last_peered":"2024-09-22T01:04:10.919226+0000","last_clean":"2024-09-22T01:04:10.919226+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:04:10.919226+0000","last_undegraded":"2024-09-22T01:04:10.919226+0000","last_fullsized":"2024-09-22T01:04:10.919226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1653,"log_dups_size":0,"ondisk_log_size":1653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:43.607282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":377,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.916415+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:04:10.916415+0000","last_peered":"2024-09-22T01:04:10.916415+0000","last_clean":"2024-09-22T01:04:10.916415+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:04:10.916415+0000","last_undegraded":"2024-09-22T01:04:10.916415+0000","last_fullsized":"2024-09-22T01:04:10.916415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'388","reported_seq":3167,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312876+0000","last_change":"2024-09-22T01:03:41.886773+0000","last_active":"2024-09-22T01:04:11.312876+0000","last_peered":"2024-09-22T01:04:11.312876+0000","last_clean":"2024-09-22T01:04:11.312876+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:04:11.312876+0000","last_undegraded":"2024-09-22T01:04:11.312876+0000","last_fullsized":"2024-09-22T01:04:11.312876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:39.279522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":390,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314477+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:04:11.314477+0000","last_peered":"2024-09-22T01:04:11.314477+0000","last_clean":"2024-09-22T01:04:11.314477+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:04:11.314477+0000","last_undegraded":"2024-09-22T01:04:11.314477+0000","last_fullsized":"2024-09-22T01:04:11.314477+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'392","reported_seq":2401,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312604+0000","last_change":"2024-09-22T01:03:41.882465+0000","last_active":"2024-09-22T01:04:11.312604+0000","last_peered":"2024-09-22T01:04:11.312604+0000","last_clean":"2024-09-22T01:04:11.312604+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:04:11.312604+0000","last_undegraded":"2024-09-22T01:04:11.312604+0000","last_fullsized":"2024-09-22T01:04:11.312604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:12.779986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314378+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:04:11.314378+0000","last_peered":"2024-09-22T01:04:11.314378+0000","last_clean":"2024-09-22T01:04:11.314378+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:04:11.314378+0000","last_undegraded":"2024-09-22T01:04:11.314378+0000","last_fullsized":"2024-09-22T01:04:11.314378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":526,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313661+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:04:11.313661+0000","last_peered":"2024-09-22T01:04:11.313661+0000","last_clean":"2024-09-22T01:04:11.313661+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:04:11.313661+0000","last_undegraded":"2024-09-22T01:04:11.313661+0000","last_fullsized":"2024-09-22T01:04:11.313661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'487","reported_seq":10794,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312452+0000","last_change":"2024-09-22T01:03:41.883318+0000","last_active":"2024-09-22T01:04:11.312452+0000","last_peered":"2024-09-22T01:04:11.312452+0000","last_clean":"2024-09-22T01:04:11.312452+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:04:11.312452+0000","last_undegraded":"2024-09-22T01:04:11.312452+0000","last_fullsized":"2024-09-22T01:04:11.312452+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:37.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'603","reported_seq":2590,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912974+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:19.230227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'801","reported_seq":16464,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315079+0000","last_change":"2024-09-22T01:03:41.893384+0000","last_active":"2024-09-22T01:04:11.315079+0000","last_peered":"2024-09-22T01:04:11.315079+0000","last_clean":"2024-09-22T01:04:11.315079+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:04:11.315079+0000","last_undegraded":"2024-09-22T01:04:11.315079+0000","last_fullsized":"2024-09-22T01:04:11.315079+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:18.880496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"175'701","reported_seq":2989,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312731+0000","last_change":"2024-09-22T01:03:41.889194+0000","last_active":"2024-09-22T01:04:11.312731+0000","last_peered":"2024-09-22T01:04:11.312731+0000","last_clean":"2024-09-22T01:04:11.312731+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:04:11.312731+0000","last_undegraded":"2024-09-22T01:04:11.312731+0000","last_fullsized":"2024-09-22T01:04:11.312731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:52.453227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'418","reported_seq":1591,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314308+0000","last_change":"2024-09-22T01:03:41.906335+0000","last_active":"2024-09-22T01:04:11.314308+0000","last_peered":"2024-09-22T01:04:11.314308+0000","last_clean":"2024-09-22T01:04:11.314308+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:04:11.314308+0000","last_undegraded":"2024-09-22T01:04:11.314308+0000","last_fullsized":"2024-09-22T01:04:11.314308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:26.787651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'759","reported_seq":6026,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313835+0000","last_change":"2024-09-22T01:03:41.999171+0000","last_active":"2024-09-22T01:04:11.313835+0000","last_peered":"2024-09-22T01:04:11.313835+0000","last_clean":"2024-09-22T01:04:11.313835+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:04:11.313835+0000","last_undegraded":"2024-09-22T01:04:11.313835+0000","last_fullsized":"2024-09-22T01:04:11.313835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:25.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'486","reported_seq":5220,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314145+0000","last_change":"2024-09-22T01:03:42.000884+0000","last_active":"2024-09-22T01:04:11.314145+0000","last_peered":"2024-09-22T01:04:11.314145+0000","last_clean":"2024-09-22T01:04:11.314145+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:04:11.314145+0000","last_undegraded":"2024-09-22T01:04:11.314145+0000","last_fullsized":"2024-09-22T01:04:11.314145+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:01.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"174'399","reported_seq":5235,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313998+0000","last_change":"2024-09-22T01:03:42.001967+0000","last_active":"2024-09-22T01:04:11.313998+0000","last_peered":"2024-09-22T01:04:11.313998+0000","last_clean":"2024-09-22T01:04:11.313998+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:04:11.313998+0000","last_undegraded":"2024-09-22T01:04:11.313998+0000","last_fullsized":"2024-09-22T01:04:11.313998+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:14.538603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'462","reported_seq":3412,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315076+0000","last_change":"2024-09-22T01:03:41.893276+0000","last_active":"2024-09-22T01:04:11.315076+0000","last_peered":"2024-09-22T01:04:11.315076+0000","last_clean":"2024-09-22T01:04:11.315076+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:04:11.315076+0000","last_undegraded":"2024-09-22T01:04:11.315076+0000","last_fullsized":"2024-09-22T01:04:11.315076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:32.260241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'680","reported_seq":3066,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313567+0000","last_change":"2024-09-22T01:03:41.997899+0000","last_active":"2024-09-22T01:04:11.313567+0000","last_peered":"2024-09-22T01:04:11.313567+0000","last_clean":"2024-09-22T01:04:11.313567+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:04:11.313567+0000","last_undegraded":"2024-09-22T01:04:11.313567+0000","last_fullsized":"2024-09-22T01:04:11.313567+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.873636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'445","reported_seq":3271,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312929+0000","last_change":"2024-09-22T01:03:41.888133+0000","last_active":"2024-09-22T01:04:11.312929+0000","last_peered":"2024-09-22T01:04:11.312929+0000","last_clean":"2024-09-22T01:04:11.312929+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:04:11.312929+0000","last_undegraded":"2024-09-22T01:04:11.312929+0000","last_fullsized":"2024-09-22T01:04:11.312929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:14.342554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'314","reported_seq":6900,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312720+0000","last_change":"2024-09-22T01:03:41.881304+0000","last_active":"2024-09-22T01:04:11.312720+0000","last_peered":"2024-09-22T01:04:11.312720+0000","last_clean":"2024-09-22T01:04:11.312720+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:04:11.312720+0000","last_undegraded":"2024-09-22T01:04:11.312720+0000","last_fullsized":"2024-09-22T01:04:11.312720+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.277966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19947,"ondisk_log_size":19947,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33548,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338292,"statfs":{"total":96636764160,"available":96602411008,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299427,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":2344,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":2400256,"data_stored":1158550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55752,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316088,"statfs":{"total":96636764160,"available":96579674112,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47432,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324408,"statfs":{"total":96636764160,"available":96588193792,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51604,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320236,"statfs":{"total":96636764160,"available":96583921664,"internally_reserved":0,"allocated":2416640,"data_stored":1164498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48008,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323832,"statfs":{"total":96636764160,"available":96587603968,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:04:15.128 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T01:04:15.129 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-22T01:04:15.439 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-22T01:04:15.451 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T01:04:15.452 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:04:15.749 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:15.749 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:04:15.765 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-22T01:04:14.676383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20067,"ondisk_log_size":20067,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378892,"kb_used_data":19980,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754595828,"statfs":{"total":773094113280,"available":772706127872,"internally_reserved":0,"allocated":20459520,"data_stored":10490563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001700"},"pg_stats":[{"pgid":"3.1f","version":"176'532","reported_seq":7206,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313143+0000","last_change":"2024-09-22T01:03:41.888720+0000","last_active":"2024-09-22T01:04:11.313143+0000","last_peered":"2024-09-22T01:04:11.313143+0000","last_clean":"2024-09-22T01:04:11.313143+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:04:11.313143+0000","last_undegraded":"2024-09-22T01:04:11.313143+0000","last_fullsized":"2024-09-22T01:04:11.313143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:31.136135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"176'811","reported_seq":3869,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312644+0000","last_change":"2024-09-22T01:03:41.885281+0000","last_active":"2024-09-22T01:04:11.312644+0000","last_peered":"2024-09-22T01:04:11.312644+0000","last_clean":"2024-09-22T01:04:11.312644+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:04:11.312644+0000","last_undegraded":"2024-09-22T01:04:11.312644+0000","last_fullsized":"2024-09-22T01:04:11.312644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:49.748988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'502","reported_seq":5361,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313981+0000","last_change":"2024-09-22T01:03:41.907575+0000","last_active":"2024-09-22T01:04:11.313981+0000","last_peered":"2024-09-22T01:04:11.313981+0000","last_clean":"2024-09-22T01:04:11.313981+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:04:11.313981+0000","last_undegraded":"2024-09-22T01:04:11.313981+0000","last_fullsized":"2024-09-22T01:04:11.313981+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:35:17.317230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1126","reported_seq":10745,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314011+0000","last_change":"2024-09-22T01:03:41.905727+0000","last_active":"2024-09-22T01:04:11.314011+0000","last_peered":"2024-09-22T01:04:11.314011+0000","last_clean":"2024-09-22T01:04:11.314011+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:04:11.314011+0000","last_undegraded":"2024-09-22T01:04:11.314011+0000","last_fullsized":"2024-09-22T01:04:11.314011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:36.367987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'1098","reported_seq":3143,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313103+0000","last_change":"2024-09-22T01:03:41.889650+0000","last_active":"2024-09-22T01:04:11.313103+0000","last_peered":"2024-09-22T01:04:11.313103+0000","last_clean":"2024-09-22T01:04:11.313103+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:04:11.313103+0000","last_undegraded":"2024-09-22T01:04:11.313103+0000","last_fullsized":"2024-09-22T01:04:11.313103+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:09.100865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'484","reported_seq":12400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912630+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:30:44.630235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'401","reported_seq":10093,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919216+0000","last_change":"2024-09-22T01:03:41.888321+0000","last_active":"2024-09-22T01:04:10.919216+0000","last_peered":"2024-09-22T01:04:10.919216+0000","last_clean":"2024-09-22T01:04:10.919216+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:04:10.919216+0000","last_undegraded":"2024-09-22T01:04:10.919216+0000","last_fullsized":"2024-09-22T01:04:10.919216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:18.365366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"176'793","reported_seq":6641,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312433+0000","last_change":"2024-09-22T01:03:41.882274+0000","last_active":"2024-09-22T01:04:11.312433+0000","last_peered":"2024-09-22T01:04:11.312433+0000","last_clean":"2024-09-22T01:04:11.312433+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:04:11.312433+0000","last_undegraded":"2024-09-22T01:04:11.312433+0000","last_fullsized":"2024-09-22T01:04:11.312433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:12.860843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'563","reported_seq":4411,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315369+0000","last_change":"2024-09-22T01:03:41.894428+0000","last_active":"2024-09-22T01:04:11.315369+0000","last_peered":"2024-09-22T01:04:11.315369+0000","last_clean":"2024-09-22T01:04:11.315369+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:04:11.315369+0000","last_undegraded":"2024-09-22T01:04:11.315369+0000","last_fullsized":"2024-09-22T01:04:11.315369+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:55.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"175'369","reported_seq":2285,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314632+0000","last_change":"2024-09-22T01:03:41.907362+0000","last_active":"2024-09-22T01:04:11.314632+0000","last_peered":"2024-09-22T01:04:11.314632+0000","last_clean":"2024-09-22T01:04:11.314632+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:04:11.314632+0000","last_undegraded":"2024-09-22T01:04:11.314632+0000","last_fullsized":"2024-09-22T01:04:11.314632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:08.940607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'687","reported_seq":6885,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313654+0000","last_change":"2024-09-22T01:03:41.999115+0000","last_active":"2024-09-22T01:04:11.313654+0000","last_peered":"2024-09-22T01:04:11.313654+0000","last_clean":"2024-09-22T01:04:11.313654+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:04:11.313654+0000","last_undegraded":"2024-09-22T01:04:11.313654+0000","last_fullsized":"2024-09-22T01:04:11.313654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:33.921063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'563","reported_seq":13986,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918873+0000","last_change":"2024-09-22T01:03:41.912104+0000","last_active":"2024-09-22T01:04:10.918873+0000","last_peered":"2024-09-22T01:04:10.918873+0000","last_clean":"2024-09-22T01:04:10.918873+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:04:10.918873+0000","last_undegraded":"2024-09-22T01:04:10.918873+0000","last_fullsized":"2024-09-22T01:04:10.918873+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:42.625118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":376,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315371+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:04:11.315371+0000","last_peered":"2024-09-22T01:04:11.315371+0000","last_clean":"2024-09-22T01:04:11.315371+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:04:11.315371+0000","last_undegraded":"2024-09-22T01:04:11.315371+0000","last_fullsized":"2024-09-22T01:04:11.315371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"176'605","reported_seq":3321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313152+0000","last_change":"2024-09-22T01:03:41.887491+0000","last_active":"2024-09-22T01:04:11.313152+0000","last_peered":"2024-09-22T01:04:11.313152+0000","last_clean":"2024-09-22T01:04:11.313152+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:04:11.313152+0000","last_undegraded":"2024-09-22T01:04:11.313152+0000","last_fullsized":"2024-09-22T01:04:11.313152+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:31.037935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919602+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:04:10.919602+0000","last_peered":"2024-09-22T01:04:10.919602+0000","last_clean":"2024-09-22T01:04:10.919602+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:04:10.919602+0000","last_undegraded":"2024-09-22T01:04:10.919602+0000","last_fullsized":"2024-09-22T01:04:10.919602+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'565","reported_seq":3653,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312775+0000","last_change":"2024-09-22T01:03:41.886088+0000","last_active":"2024-09-22T01:04:11.312775+0000","last_peered":"2024-09-22T01:04:11.312775+0000","last_clean":"2024-09-22T01:04:11.312775+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:04:11.312775+0000","last_undegraded":"2024-09-22T01:04:11.312775+0000","last_fullsized":"2024-09-22T01:04:11.312775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:49.543414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":375,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313917+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:04:11.313917+0000","last_peered":"2024-09-22T01:04:11.313917+0000","last_clean":"2024-09-22T01:04:11.313917+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:04:11.313917+0000","last_undegraded":"2024-09-22T01:04:11.313917+0000","last_fullsized":"2024-09-22T01:04:11.313917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'444","reported_seq":4520,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919765+0000","last_change":"2024-09-22T01:03:41.883039+0000","last_active":"2024-09-22T01:04:10.919765+0000","last_peered":"2024-09-22T01:04:10.919765+0000","last_clean":"2024-09-22T01:04:10.919765+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:04:10.919765+0000","last_undegraded":"2024-09-22T01:04:10.919765+0000","last_fullsized":"2024-09-22T01:04:10.919765+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:52.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919513+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:04:10.919513+0000","last_peered":"2024-09-22T01:04:10.919513+0000","last_clean":"2024-09-22T01:04:10.919513+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:04:10.919513+0000","last_undegraded":"2024-09-22T01:04:10.919513+0000","last_fullsized":"2024-09-22T01:04:10.919513+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'680","reported_seq":4687,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314274+0000","last_change":"2024-09-22T01:03:41.906860+0000","last_active":"2024-09-22T01:04:11.314274+0000","last_peered":"2024-09-22T01:04:11.314274+0000","last_clean":"2024-09-22T01:04:11.314274+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:04:11.314274+0000","last_undegraded":"2024-09-22T01:04:11.314274+0000","last_fullsized":"2024-09-22T01:04:11.314274+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:43.965472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"176'736","reported_seq":3097,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918646+0000","last_change":"2024-09-22T01:03:41.913492+0000","last_active":"2024-09-22T01:04:10.918646+0000","last_peered":"2024-09-22T01:04:10.918646+0000","last_clean":"2024-09-22T01:04:10.918646+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:04:10.918646+0000","last_undegraded":"2024-09-22T01:04:10.918646+0000","last_fullsized":"2024-09-22T01:04:10.918646+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:27.079591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":349,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314672+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:04:11.314672+0000","last_peered":"2024-09-22T01:04:11.314672+0000","last_clean":"2024-09-22T01:04:11.314672+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:04:11.314672+0000","last_undegraded":"2024-09-22T01:04:11.314672+0000","last_fullsized":"2024-09-22T01:04:11.314672+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'1653","reported_seq":3951,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919226+0000","last_change":"2024-09-22T01:03:41.883040+0000","last_active":"2024-09-22T01:04:10.919226+0000","last_peered":"2024-09-22T01:04:10.919226+0000","last_clean":"2024-09-22T01:04:10.919226+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:04:10.919226+0000","last_undegraded":"2024-09-22T01:04:10.919226+0000","last_fullsized":"2024-09-22T01:04:10.919226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1653,"log_dups_size":0,"ondisk_log_size":1653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:43.607282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":377,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.916415+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:04:10.916415+0000","last_peered":"2024-09-22T01:04:10.916415+0000","last_clean":"2024-09-22T01:04:10.916415+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:04:10.916415+0000","last_undegraded":"2024-09-22T01:04:10.916415+0000","last_fullsized":"2024-09-22T01:04:10.916415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'388","reported_seq":3167,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312876+0000","last_change":"2024-09-22T01:03:41.886773+0000","last_active":"2024-09-22T01:04:11.312876+0000","last_peered":"2024-09-22T01:04:11.312876+0000","last_clean":"2024-09-22T01:04:11.312876+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:04:11.312876+0000","last_undegraded":"2024-09-22T01:04:11.312876+0000","last_fullsized":"2024-09-22T01:04:11.312876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:39.279522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":390,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314477+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:04:11.314477+0000","last_peered":"2024-09-22T01:04:11.314477+0000","last_clean":"2024-09-22T01:04:11.314477+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:04:11.314477+0000","last_undegraded":"2024-09-22T01:04:11.314477+0000","last_fullsized":"2024-09-22T01:04:11.314477+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'392","reported_seq":2401,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312604+0000","last_change":"2024-09-22T01:03:41.882465+0000","last_active":"2024-09-22T01:04:11.312604+0000","last_peered":"2024-09-22T01:04:11.312604+0000","last_clean":"2024-09-22T01:04:11.312604+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:04:11.312604+0000","last_undegraded":"2024-09-22T01:04:11.312604+0000","last_fullsized":"2024-09-22T01:04:11.312604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:12.779986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314378+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:04:11.314378+0000","last_peered":"2024-09-22T01:04:11.314378+0000","last_clean":"2024-09-22T01:04:11.314378+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:04:11.314378+0000","last_undegraded":"2024-09-22T01:04:11.314378+0000","last_fullsized":"2024-09-22T01:04:11.314378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":526,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313661+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:04:11.313661+0000","last_peered":"2024-09-22T01:04:11.313661+0000","last_clean":"2024-09-22T01:04:11.313661+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:04:11.313661+0000","last_undegraded":"2024-09-22T01:04:11.313661+0000","last_fullsized":"2024-09-22T01:04:11.313661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'487","reported_seq":10794,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312452+0000","last_change":"2024-09-22T01:03:41.883318+0000","last_active":"2024-09-22T01:04:11.312452+0000","last_peered":"2024-09-22T01:04:11.312452+0000","last_clean":"2024-09-22T01:04:11.312452+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:04:11.312452+0000","last_undegraded":"2024-09-22T01:04:11.312452+0000","last_fullsized":"2024-09-22T01:04:11.312452+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:37.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'603","reported_seq":2590,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912974+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:19.230227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'801","reported_seq":16464,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315079+0000","last_change":"2024-09-22T01:03:41.893384+0000","last_active":"2024-09-22T01:04:11.315079+0000","last_peered":"2024-09-22T01:04:11.315079+0000","last_clean":"2024-09-22T01:04:11.315079+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:04:11.315079+0000","last_undegraded":"2024-09-22T01:04:11.315079+0000","last_fullsized":"2024-09-22T01:04:11.315079+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:18.880496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"175'701","reported_seq":2989,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312731+0000","last_change":"2024-09-22T01:03:41.889194+0000","last_active":"2024-09-22T01:04:11.312731+0000","last_peered":"2024-09-22T01:04:11.312731+0000","last_clean":"2024-09-22T01:04:11.312731+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:04:11.312731+0000","last_undegraded":"2024-09-22T01:04:11.312731+0000","last_fullsized":"2024-09-22T01:04:11.312731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:52.453227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'418","reported_seq":1591,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314308+0000","last_change":"2024-09-22T01:03:41.906335+0000","last_active":"2024-09-22T01:04:11.314308+0000","last_peered":"2024-09-22T01:04:11.314308+0000","last_clean":"2024-09-22T01:04:11.314308+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:04:11.314308+0000","last_undegraded":"2024-09-22T01:04:11.314308+0000","last_fullsized":"2024-09-22T01:04:11.314308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:26.787651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'759","reported_seq":6026,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313835+0000","last_change":"2024-09-22T01:03:41.999171+0000","last_active":"2024-09-22T01:04:11.313835+0000","last_peered":"2024-09-22T01:04:11.313835+0000","last_clean":"2024-09-22T01:04:11.313835+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:04:11.313835+0000","last_undegraded":"2024-09-22T01:04:11.313835+0000","last_fullsized":"2024-09-22T01:04:11.313835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:25.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'486","reported_seq":5220,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314145+0000","last_change":"2024-09-22T01:03:42.000884+0000","last_active":"2024-09-22T01:04:11.314145+0000","last_peered":"2024-09-22T01:04:11.314145+0000","last_clean":"2024-09-22T01:04:11.314145+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:04:11.314145+0000","last_undegraded":"2024-09-22T01:04:11.314145+0000","last_fullsized":"2024-09-22T01:04:11.314145+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:01.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"174'399","reported_seq":5235,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313998+0000","last_change":"2024-09-22T01:03:42.001967+0000","last_active":"2024-09-22T01:04:11.313998+0000","last_peered":"2024-09-22T01:04:11.313998+0000","last_clean":"2024-09-22T01:04:11.313998+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:04:11.313998+0000","last_undegraded":"2024-09-22T01:04:11.313998+0000","last_fullsized":"2024-09-22T01:04:11.313998+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:14.538603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'462","reported_seq":3412,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315076+0000","last_change":"2024-09-22T01:03:41.893276+0000","last_active":"2024-09-22T01:04:11.315076+0000","last_peered":"2024-09-22T01:04:11.315076+0000","last_clean":"2024-09-22T01:04:11.315076+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:04:11.315076+0000","last_undegraded":"2024-09-22T01:04:11.315076+0000","last_fullsized":"2024-09-22T01:04:11.315076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:32.260241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'680","reported_seq":3066,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313567+0000","last_change":"2024-09-22T01:03:41.997899+0000","last_active":"2024-09-22T01:04:11.313567+0000","last_peered":"2024-09-22T01:04:11.313567+0000","last_clean":"2024-09-22T01:04:11.313567+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:04:11.313567+0000","last_undegraded":"2024-09-22T01:04:11.313567+0000","last_fullsized":"2024-09-22T01:04:11.313567+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.873636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'445","reported_seq":3271,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312929+0000","last_change":"2024-09-22T01:03:41.888133+0000","last_active":"2024-09-22T01:04:11.312929+0000","last_peered":"2024-09-22T01:04:11.312929+0000","last_clean":"2024-09-22T01:04:11.312929+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:04:11.312929+0000","last_undegraded":"2024-09-22T01:04:11.312929+0000","last_fullsized":"2024-09-22T01:04:11.312929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:14.342554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'314","reported_seq":6900,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312720+0000","last_change":"2024-09-22T01:03:41.881304+0000","last_active":"2024-09-22T01:04:11.312720+0000","last_peered":"2024-09-22T01:04:11.312720+0000","last_clean":"2024-09-22T01:04:11.312720+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:04:11.312720+0000","last_undegraded":"2024-09-22T01:04:11.312720+0000","last_fullsized":"2024-09-22T01:04:11.312720+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.277966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19947,"ondisk_log_size":19947,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33548,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338292,"statfs":{"total":96636764160,"available":96602411008,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299427,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":2344,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":2400256,"data_stored":1158550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55752,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316088,"statfs":{"total":96636764160,"available":96579674112,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47432,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324408,"statfs":{"total":96636764160,"available":96588193792,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51604,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320236,"statfs":{"total":96636764160,"available":96583921664,"internally_reserved":0,"allocated":2416640,"data_stored":1164498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48008,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323832,"statfs":{"total":96636764160,"available":96587603968,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:04:15.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:04:16.066 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:04:16.083 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-22T01:04:14.676383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20067,"ondisk_log_size":20067,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378892,"kb_used_data":19980,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754595828,"statfs":{"total":773094113280,"available":772706127872,"internally_reserved":0,"allocated":20459520,"data_stored":10490563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001700"},"pg_stats":[{"pgid":"3.1f","version":"176'532","reported_seq":7206,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313143+0000","last_change":"2024-09-22T01:03:41.888720+0000","last_active":"2024-09-22T01:04:11.313143+0000","last_peered":"2024-09-22T01:04:11.313143+0000","last_clean":"2024-09-22T01:04:11.313143+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:04:11.313143+0000","last_undegraded":"2024-09-22T01:04:11.313143+0000","last_fullsized":"2024-09-22T01:04:11.313143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:31.136135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"176'811","reported_seq":3869,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312644+0000","last_change":"2024-09-22T01:03:41.885281+0000","last_active":"2024-09-22T01:04:11.312644+0000","last_peered":"2024-09-22T01:04:11.312644+0000","last_clean":"2024-09-22T01:04:11.312644+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:04:11.312644+0000","last_undegraded":"2024-09-22T01:04:11.312644+0000","last_fullsized":"2024-09-22T01:04:11.312644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:49.748988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'502","reported_seq":5361,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313981+0000","last_change":"2024-09-22T01:03:41.907575+0000","last_active":"2024-09-22T01:04:11.313981+0000","last_peered":"2024-09-22T01:04:11.313981+0000","last_clean":"2024-09-22T01:04:11.313981+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:04:11.313981+0000","last_undegraded":"2024-09-22T01:04:11.313981+0000","last_fullsized":"2024-09-22T01:04:11.313981+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:35:17.317230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1126","reported_seq":10745,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314011+0000","last_change":"2024-09-22T01:03:41.905727+0000","last_active":"2024-09-22T01:04:11.314011+0000","last_peered":"2024-09-22T01:04:11.314011+0000","last_clean":"2024-09-22T01:04:11.314011+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:04:11.314011+0000","last_undegraded":"2024-09-22T01:04:11.314011+0000","last_fullsized":"2024-09-22T01:04:11.314011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:36.367987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'1098","reported_seq":3143,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313103+0000","last_change":"2024-09-22T01:03:41.889650+0000","last_active":"2024-09-22T01:04:11.313103+0000","last_peered":"2024-09-22T01:04:11.313103+0000","last_clean":"2024-09-22T01:04:11.313103+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:04:11.313103+0000","last_undegraded":"2024-09-22T01:04:11.313103+0000","last_fullsized":"2024-09-22T01:04:11.313103+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:09.100865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'484","reported_seq":12400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912630+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:30:44.630235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'401","reported_seq":10093,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919216+0000","last_change":"2024-09-22T01:03:41.888321+0000","last_active":"2024-09-22T01:04:10.919216+0000","last_peered":"2024-09-22T01:04:10.919216+0000","last_clean":"2024-09-22T01:04:10.919216+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:04:10.919216+0000","last_undegraded":"2024-09-22T01:04:10.919216+0000","last_fullsized":"2024-09-22T01:04:10.919216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:18.365366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"176'793","reported_seq":6641,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312433+0000","last_change":"2024-09-22T01:03:41.882274+0000","last_active":"2024-09-22T01:04:11.312433+0000","last_peered":"2024-09-22T01:04:11.312433+0000","last_clean":"2024-09-22T01:04:11.312433+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:04:11.312433+0000","last_undegraded":"2024-09-22T01:04:11.312433+0000","last_fullsized":"2024-09-22T01:04:11.312433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:12.860843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'563","reported_seq":4411,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315369+0000","last_change":"2024-09-22T01:03:41.894428+0000","last_active":"2024-09-22T01:04:11.315369+0000","last_peered":"2024-09-22T01:04:11.315369+0000","last_clean":"2024-09-22T01:04:11.315369+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:04:11.315369+0000","last_undegraded":"2024-09-22T01:04:11.315369+0000","last_fullsized":"2024-09-22T01:04:11.315369+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:55.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"175'369","reported_seq":2285,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314632+0000","last_change":"2024-09-22T01:03:41.907362+0000","last_active":"2024-09-22T01:04:11.314632+0000","last_peered":"2024-09-22T01:04:11.314632+0000","last_clean":"2024-09-22T01:04:11.314632+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:04:11.314632+0000","last_undegraded":"2024-09-22T01:04:11.314632+0000","last_fullsized":"2024-09-22T01:04:11.314632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:08.940607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'687","reported_seq":6885,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313654+0000","last_change":"2024-09-22T01:03:41.999115+0000","last_active":"2024-09-22T01:04:11.313654+0000","last_peered":"2024-09-22T01:04:11.313654+0000","last_clean":"2024-09-22T01:04:11.313654+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:04:11.313654+0000","last_undegraded":"2024-09-22T01:04:11.313654+0000","last_fullsized":"2024-09-22T01:04:11.313654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:33.921063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'563","reported_seq":13986,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918873+0000","last_change":"2024-09-22T01:03:41.912104+0000","last_active":"2024-09-22T01:04:10.918873+0000","last_peered":"2024-09-22T01:04:10.918873+0000","last_clean":"2024-09-22T01:04:10.918873+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:04:10.918873+0000","last_undegraded":"2024-09-22T01:04:10.918873+0000","last_fullsized":"2024-09-22T01:04:10.918873+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:42.625118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":376,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315371+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:04:11.315371+0000","last_peered":"2024-09-22T01:04:11.315371+0000","last_clean":"2024-09-22T01:04:11.315371+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:04:11.315371+0000","last_undegraded":"2024-09-22T01:04:11.315371+0000","last_fullsized":"2024-09-22T01:04:11.315371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"176'605","reported_seq":3321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313152+0000","last_change":"2024-09-22T01:03:41.887491+0000","last_active":"2024-09-22T01:04:11.313152+0000","last_peered":"2024-09-22T01:04:11.313152+0000","last_clean":"2024-09-22T01:04:11.313152+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:04:11.313152+0000","last_undegraded":"2024-09-22T01:04:11.313152+0000","last_fullsized":"2024-09-22T01:04:11.313152+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:31.037935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919602+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:04:10.919602+0000","last_peered":"2024-09-22T01:04:10.919602+0000","last_clean":"2024-09-22T01:04:10.919602+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:04:10.919602+0000","last_undegraded":"2024-09-22T01:04:10.919602+0000","last_fullsized":"2024-09-22T01:04:10.919602+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'565","reported_seq":3653,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312775+0000","last_change":"2024-09-22T01:03:41.886088+0000","last_active":"2024-09-22T01:04:11.312775+0000","last_peered":"2024-09-22T01:04:11.312775+0000","last_clean":"2024-09-22T01:04:11.312775+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:04:11.312775+0000","last_undegraded":"2024-09-22T01:04:11.312775+0000","last_fullsized":"2024-09-22T01:04:11.312775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:49.543414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":375,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313917+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:04:11.313917+0000","last_peered":"2024-09-22T01:04:11.313917+0000","last_clean":"2024-09-22T01:04:11.313917+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:04:11.313917+0000","last_undegraded":"2024-09-22T01:04:11.313917+0000","last_fullsized":"2024-09-22T01:04:11.313917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'444","reported_seq":4520,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919765+0000","last_change":"2024-09-22T01:03:41.883039+0000","last_active":"2024-09-22T01:04:10.919765+0000","last_peered":"2024-09-22T01:04:10.919765+0000","last_clean":"2024-09-22T01:04:10.919765+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:04:10.919765+0000","last_undegraded":"2024-09-22T01:04:10.919765+0000","last_fullsized":"2024-09-22T01:04:10.919765+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:52.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919513+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:04:10.919513+0000","last_peered":"2024-09-22T01:04:10.919513+0000","last_clean":"2024-09-22T01:04:10.919513+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:04:10.919513+0000","last_undegraded":"2024-09-22T01:04:10.919513+0000","last_fullsized":"2024-09-22T01:04:10.919513+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'680","reported_seq":4687,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314274+0000","last_change":"2024-09-22T01:03:41.906860+0000","last_active":"2024-09-22T01:04:11.314274+0000","last_peered":"2024-09-22T01:04:11.314274+0000","last_clean":"2024-09-22T01:04:11.314274+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:04:11.314274+0000","last_undegraded":"2024-09-22T01:04:11.314274+0000","last_fullsized":"2024-09-22T01:04:11.314274+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:43.965472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"176'736","reported_seq":3097,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918646+0000","last_change":"2024-09-22T01:03:41.913492+0000","last_active":"2024-09-22T01:04:10.918646+0000","last_peered":"2024-09-22T01:04:10.918646+0000","last_clean":"2024-09-22T01:04:10.918646+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:04:10.918646+0000","last_undegraded":"2024-09-22T01:04:10.918646+0000","last_fullsized":"2024-09-22T01:04:10.918646+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:27.079591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":349,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314672+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:04:11.314672+0000","last_peered":"2024-09-22T01:04:11.314672+0000","last_clean":"2024-09-22T01:04:11.314672+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:04:11.314672+0000","last_undegraded":"2024-09-22T01:04:11.314672+0000","last_fullsized":"2024-09-22T01:04:11.314672+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'1653","reported_seq":3951,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919226+0000","last_change":"2024-09-22T01:03:41.883040+0000","last_active":"2024-09-22T01:04:10.919226+0000","last_peered":"2024-09-22T01:04:10.919226+0000","last_clean":"2024-09-22T01:04:10.919226+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:04:10.919226+0000","last_undegraded":"2024-09-22T01:04:10.919226+0000","last_fullsized":"2024-09-22T01:04:10.919226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1653,"log_dups_size":0,"ondisk_log_size":1653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:43.607282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":377,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.916415+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:04:10.916415+0000","last_peered":"2024-09-22T01:04:10.916415+0000","last_clean":"2024-09-22T01:04:10.916415+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:04:10.916415+0000","last_undegraded":"2024-09-22T01:04:10.916415+0000","last_fullsized":"2024-09-22T01:04:10.916415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'388","reported_seq":3167,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312876+0000","last_change":"2024-09-22T01:03:41.886773+0000","last_active":"2024-09-22T01:04:11.312876+0000","last_peered":"2024-09-22T01:04:11.312876+0000","last_clean":"2024-09-22T01:04:11.312876+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:04:11.312876+0000","last_undegraded":"2024-09-22T01:04:11.312876+0000","last_fullsized":"2024-09-22T01:04:11.312876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:39.279522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":390,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314477+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:04:11.314477+0000","last_peered":"2024-09-22T01:04:11.314477+0000","last_clean":"2024-09-22T01:04:11.314477+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:04:11.314477+0000","last_undegraded":"2024-09-22T01:04:11.314477+0000","last_fullsized":"2024-09-22T01:04:11.314477+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'392","reported_seq":2401,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312604+0000","last_change":"2024-09-22T01:03:41.882465+0000","last_active":"2024-09-22T01:04:11.312604+0000","last_peered":"2024-09-22T01:04:11.312604+0000","last_clean":"2024-09-22T01:04:11.312604+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:04:11.312604+0000","last_undegraded":"2024-09-22T01:04:11.312604+0000","last_fullsized":"2024-09-22T01:04:11.312604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:12.779986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314378+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:04:11.314378+0000","last_peered":"2024-09-22T01:04:11.314378+0000","last_clean":"2024-09-22T01:04:11.314378+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:04:11.314378+0000","last_undegraded":"2024-09-22T01:04:11.314378+0000","last_fullsized":"2024-09-22T01:04:11.314378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":526,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313661+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:04:11.313661+0000","last_peered":"2024-09-22T01:04:11.313661+0000","last_clean":"2024-09-22T01:04:11.313661+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:04:11.313661+0000","last_undegraded":"2024-09-22T01:04:11.313661+0000","last_fullsized":"2024-09-22T01:04:11.313661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'487","reported_seq":10794,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312452+0000","last_change":"2024-09-22T01:03:41.883318+0000","last_active":"2024-09-22T01:04:11.312452+0000","last_peered":"2024-09-22T01:04:11.312452+0000","last_clean":"2024-09-22T01:04:11.312452+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:04:11.312452+0000","last_undegraded":"2024-09-22T01:04:11.312452+0000","last_fullsized":"2024-09-22T01:04:11.312452+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:37.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'603","reported_seq":2590,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912974+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:19.230227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'801","reported_seq":16464,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315079+0000","last_change":"2024-09-22T01:03:41.893384+0000","last_active":"2024-09-22T01:04:11.315079+0000","last_peered":"2024-09-22T01:04:11.315079+0000","last_clean":"2024-09-22T01:04:11.315079+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:04:11.315079+0000","last_undegraded":"2024-09-22T01:04:11.315079+0000","last_fullsized":"2024-09-22T01:04:11.315079+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:18.880496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"175'701","reported_seq":2989,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312731+0000","last_change":"2024-09-22T01:03:41.889194+0000","last_active":"2024-09-22T01:04:11.312731+0000","last_peered":"2024-09-22T01:04:11.312731+0000","last_clean":"2024-09-22T01:04:11.312731+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:04:11.312731+0000","last_undegraded":"2024-09-22T01:04:11.312731+0000","last_fullsized":"2024-09-22T01:04:11.312731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:52.453227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'418","reported_seq":1591,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314308+0000","last_change":"2024-09-22T01:03:41.906335+0000","last_active":"2024-09-22T01:04:11.314308+0000","last_peered":"2024-09-22T01:04:11.314308+0000","last_clean":"2024-09-22T01:04:11.314308+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:04:11.314308+0000","last_undegraded":"2024-09-22T01:04:11.314308+0000","last_fullsized":"2024-09-22T01:04:11.314308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:26.787651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'759","reported_seq":6026,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313835+0000","last_change":"2024-09-22T01:03:41.999171+0000","last_active":"2024-09-22T01:04:11.313835+0000","last_peered":"2024-09-22T01:04:11.313835+0000","last_clean":"2024-09-22T01:04:11.313835+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:04:11.313835+0000","last_undegraded":"2024-09-22T01:04:11.313835+0000","last_fullsized":"2024-09-22T01:04:11.313835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:25.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'486","reported_seq":5220,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314145+0000","last_change":"2024-09-22T01:03:42.000884+0000","last_active":"2024-09-22T01:04:11.314145+0000","last_peered":"2024-09-22T01:04:11.314145+0000","last_clean":"2024-09-22T01:04:11.314145+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:04:11.314145+0000","last_undegraded":"2024-09-22T01:04:11.314145+0000","last_fullsized":"2024-09-22T01:04:11.314145+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:01.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"174'399","reported_seq":5235,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313998+0000","last_change":"2024-09-22T01:03:42.001967+0000","last_active":"2024-09-22T01:04:11.313998+0000","last_peered":"2024-09-22T01:04:11.313998+0000","last_clean":"2024-09-22T01:04:11.313998+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:04:11.313998+0000","last_undegraded":"2024-09-22T01:04:11.313998+0000","last_fullsized":"2024-09-22T01:04:11.313998+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:14.538603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'462","reported_seq":3412,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315076+0000","last_change":"2024-09-22T01:03:41.893276+0000","last_active":"2024-09-22T01:04:11.315076+0000","last_peered":"2024-09-22T01:04:11.315076+0000","last_clean":"2024-09-22T01:04:11.315076+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:04:11.315076+0000","last_undegraded":"2024-09-22T01:04:11.315076+0000","last_fullsized":"2024-09-22T01:04:11.315076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:32.260241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'680","reported_seq":3066,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313567+0000","last_change":"2024-09-22T01:03:41.997899+0000","last_active":"2024-09-22T01:04:11.313567+0000","last_peered":"2024-09-22T01:04:11.313567+0000","last_clean":"2024-09-22T01:04:11.313567+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:04:11.313567+0000","last_undegraded":"2024-09-22T01:04:11.313567+0000","last_fullsized":"2024-09-22T01:04:11.313567+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.873636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'445","reported_seq":3271,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312929+0000","last_change":"2024-09-22T01:03:41.888133+0000","last_active":"2024-09-22T01:04:11.312929+0000","last_peered":"2024-09-22T01:04:11.312929+0000","last_clean":"2024-09-22T01:04:11.312929+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:04:11.312929+0000","last_undegraded":"2024-09-22T01:04:11.312929+0000","last_fullsized":"2024-09-22T01:04:11.312929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:14.342554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'314","reported_seq":6900,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312720+0000","last_change":"2024-09-22T01:03:41.881304+0000","last_active":"2024-09-22T01:04:11.312720+0000","last_peered":"2024-09-22T01:04:11.312720+0000","last_clean":"2024-09-22T01:04:11.312720+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:04:11.312720+0000","last_undegraded":"2024-09-22T01:04:11.312720+0000","last_fullsized":"2024-09-22T01:04:11.312720+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.277966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19947,"ondisk_log_size":19947,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33548,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338292,"statfs":{"total":96636764160,"available":96602411008,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299427,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":2344,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":2400256,"data_stored":1158550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55752,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316088,"statfs":{"total":96636764160,"available":96579674112,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47432,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324408,"statfs":{"total":96636764160,"available":96588193792,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51604,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320236,"statfs":{"total":96636764160,"available":96583921664,"internally_reserved":0,"allocated":2416640,"data_stored":1164498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48008,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323832,"statfs":{"total":96636764160,"available":96587603968,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:04:16.085 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T01:04:16.086 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:04:16.379 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:04:16.397 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-22T01:04:14.676383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20067,"ondisk_log_size":20067,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378892,"kb_used_data":19980,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754595828,"statfs":{"total":773094113280,"available":772706127872,"internally_reserved":0,"allocated":20459520,"data_stored":10490563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001700"},"pg_stats":[{"pgid":"3.1f","version":"176'532","reported_seq":7206,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313143+0000","last_change":"2024-09-22T01:03:41.888720+0000","last_active":"2024-09-22T01:04:11.313143+0000","last_peered":"2024-09-22T01:04:11.313143+0000","last_clean":"2024-09-22T01:04:11.313143+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:04:11.313143+0000","last_undegraded":"2024-09-22T01:04:11.313143+0000","last_fullsized":"2024-09-22T01:04:11.313143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:31.136135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"176'811","reported_seq":3869,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312644+0000","last_change":"2024-09-22T01:03:41.885281+0000","last_active":"2024-09-22T01:04:11.312644+0000","last_peered":"2024-09-22T01:04:11.312644+0000","last_clean":"2024-09-22T01:04:11.312644+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:04:11.312644+0000","last_undegraded":"2024-09-22T01:04:11.312644+0000","last_fullsized":"2024-09-22T01:04:11.312644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:49.748988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'502","reported_seq":5361,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313981+0000","last_change":"2024-09-22T01:03:41.907575+0000","last_active":"2024-09-22T01:04:11.313981+0000","last_peered":"2024-09-22T01:04:11.313981+0000","last_clean":"2024-09-22T01:04:11.313981+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:04:11.313981+0000","last_undegraded":"2024-09-22T01:04:11.313981+0000","last_fullsized":"2024-09-22T01:04:11.313981+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:35:17.317230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1126","reported_seq":10745,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314011+0000","last_change":"2024-09-22T01:03:41.905727+0000","last_active":"2024-09-22T01:04:11.314011+0000","last_peered":"2024-09-22T01:04:11.314011+0000","last_clean":"2024-09-22T01:04:11.314011+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:04:11.314011+0000","last_undegraded":"2024-09-22T01:04:11.314011+0000","last_fullsized":"2024-09-22T01:04:11.314011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:36.367987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'1098","reported_seq":3143,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313103+0000","last_change":"2024-09-22T01:03:41.889650+0000","last_active":"2024-09-22T01:04:11.313103+0000","last_peered":"2024-09-22T01:04:11.313103+0000","last_clean":"2024-09-22T01:04:11.313103+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:04:11.313103+0000","last_undegraded":"2024-09-22T01:04:11.313103+0000","last_fullsized":"2024-09-22T01:04:11.313103+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:09.100865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'484","reported_seq":12400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912630+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:30:44.630235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'401","reported_seq":10093,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919216+0000","last_change":"2024-09-22T01:03:41.888321+0000","last_active":"2024-09-22T01:04:10.919216+0000","last_peered":"2024-09-22T01:04:10.919216+0000","last_clean":"2024-09-22T01:04:10.919216+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:04:10.919216+0000","last_undegraded":"2024-09-22T01:04:10.919216+0000","last_fullsized":"2024-09-22T01:04:10.919216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:18.365366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"176'793","reported_seq":6641,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312433+0000","last_change":"2024-09-22T01:03:41.882274+0000","last_active":"2024-09-22T01:04:11.312433+0000","last_peered":"2024-09-22T01:04:11.312433+0000","last_clean":"2024-09-22T01:04:11.312433+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:04:11.312433+0000","last_undegraded":"2024-09-22T01:04:11.312433+0000","last_fullsized":"2024-09-22T01:04:11.312433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:12.860843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'563","reported_seq":4411,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315369+0000","last_change":"2024-09-22T01:03:41.894428+0000","last_active":"2024-09-22T01:04:11.315369+0000","last_peered":"2024-09-22T01:04:11.315369+0000","last_clean":"2024-09-22T01:04:11.315369+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:04:11.315369+0000","last_undegraded":"2024-09-22T01:04:11.315369+0000","last_fullsized":"2024-09-22T01:04:11.315369+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:55.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"175'369","reported_seq":2285,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314632+0000","last_change":"2024-09-22T01:03:41.907362+0000","last_active":"2024-09-22T01:04:11.314632+0000","last_peered":"2024-09-22T01:04:11.314632+0000","last_clean":"2024-09-22T01:04:11.314632+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:04:11.314632+0000","last_undegraded":"2024-09-22T01:04:11.314632+0000","last_fullsized":"2024-09-22T01:04:11.314632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:08.940607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'687","reported_seq":6885,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313654+0000","last_change":"2024-09-22T01:03:41.999115+0000","last_active":"2024-09-22T01:04:11.313654+0000","last_peered":"2024-09-22T01:04:11.313654+0000","last_clean":"2024-09-22T01:04:11.313654+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:04:11.313654+0000","last_undegraded":"2024-09-22T01:04:11.313654+0000","last_fullsized":"2024-09-22T01:04:11.313654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:33.921063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'563","reported_seq":13986,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918873+0000","last_change":"2024-09-22T01:03:41.912104+0000","last_active":"2024-09-22T01:04:10.918873+0000","last_peered":"2024-09-22T01:04:10.918873+0000","last_clean":"2024-09-22T01:04:10.918873+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:04:10.918873+0000","last_undegraded":"2024-09-22T01:04:10.918873+0000","last_fullsized":"2024-09-22T01:04:10.918873+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:42.625118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":376,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315371+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:04:11.315371+0000","last_peered":"2024-09-22T01:04:11.315371+0000","last_clean":"2024-09-22T01:04:11.315371+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:04:11.315371+0000","last_undegraded":"2024-09-22T01:04:11.315371+0000","last_fullsized":"2024-09-22T01:04:11.315371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"176'605","reported_seq":3321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313152+0000","last_change":"2024-09-22T01:03:41.887491+0000","last_active":"2024-09-22T01:04:11.313152+0000","last_peered":"2024-09-22T01:04:11.313152+0000","last_clean":"2024-09-22T01:04:11.313152+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:04:11.313152+0000","last_undegraded":"2024-09-22T01:04:11.313152+0000","last_fullsized":"2024-09-22T01:04:11.313152+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:31.037935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919602+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:04:10.919602+0000","last_peered":"2024-09-22T01:04:10.919602+0000","last_clean":"2024-09-22T01:04:10.919602+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:04:10.919602+0000","last_undegraded":"2024-09-22T01:04:10.919602+0000","last_fullsized":"2024-09-22T01:04:10.919602+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'565","reported_seq":3653,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312775+0000","last_change":"2024-09-22T01:03:41.886088+0000","last_active":"2024-09-22T01:04:11.312775+0000","last_peered":"2024-09-22T01:04:11.312775+0000","last_clean":"2024-09-22T01:04:11.312775+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:04:11.312775+0000","last_undegraded":"2024-09-22T01:04:11.312775+0000","last_fullsized":"2024-09-22T01:04:11.312775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:49.543414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":375,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313917+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:04:11.313917+0000","last_peered":"2024-09-22T01:04:11.313917+0000","last_clean":"2024-09-22T01:04:11.313917+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:04:11.313917+0000","last_undegraded":"2024-09-22T01:04:11.313917+0000","last_fullsized":"2024-09-22T01:04:11.313917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'444","reported_seq":4520,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919765+0000","last_change":"2024-09-22T01:03:41.883039+0000","last_active":"2024-09-22T01:04:10.919765+0000","last_peered":"2024-09-22T01:04:10.919765+0000","last_clean":"2024-09-22T01:04:10.919765+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:04:10.919765+0000","last_undegraded":"2024-09-22T01:04:10.919765+0000","last_fullsized":"2024-09-22T01:04:10.919765+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:52.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919513+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:04:10.919513+0000","last_peered":"2024-09-22T01:04:10.919513+0000","last_clean":"2024-09-22T01:04:10.919513+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:04:10.919513+0000","last_undegraded":"2024-09-22T01:04:10.919513+0000","last_fullsized":"2024-09-22T01:04:10.919513+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'680","reported_seq":4687,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314274+0000","last_change":"2024-09-22T01:03:41.906860+0000","last_active":"2024-09-22T01:04:11.314274+0000","last_peered":"2024-09-22T01:04:11.314274+0000","last_clean":"2024-09-22T01:04:11.314274+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:04:11.314274+0000","last_undegraded":"2024-09-22T01:04:11.314274+0000","last_fullsized":"2024-09-22T01:04:11.314274+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:43.965472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"176'736","reported_seq":3097,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918646+0000","last_change":"2024-09-22T01:03:41.913492+0000","last_active":"2024-09-22T01:04:10.918646+0000","last_peered":"2024-09-22T01:04:10.918646+0000","last_clean":"2024-09-22T01:04:10.918646+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:04:10.918646+0000","last_undegraded":"2024-09-22T01:04:10.918646+0000","last_fullsized":"2024-09-22T01:04:10.918646+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:27.079591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":349,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314672+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:04:11.314672+0000","last_peered":"2024-09-22T01:04:11.314672+0000","last_clean":"2024-09-22T01:04:11.314672+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:04:11.314672+0000","last_undegraded":"2024-09-22T01:04:11.314672+0000","last_fullsized":"2024-09-22T01:04:11.314672+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'1653","reported_seq":3951,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919226+0000","last_change":"2024-09-22T01:03:41.883040+0000","last_active":"2024-09-22T01:04:10.919226+0000","last_peered":"2024-09-22T01:04:10.919226+0000","last_clean":"2024-09-22T01:04:10.919226+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:04:10.919226+0000","last_undegraded":"2024-09-22T01:04:10.919226+0000","last_fullsized":"2024-09-22T01:04:10.919226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1653,"log_dups_size":0,"ondisk_log_size":1653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:43.607282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":377,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.916415+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:04:10.916415+0000","last_peered":"2024-09-22T01:04:10.916415+0000","last_clean":"2024-09-22T01:04:10.916415+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:04:10.916415+0000","last_undegraded":"2024-09-22T01:04:10.916415+0000","last_fullsized":"2024-09-22T01:04:10.916415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'388","reported_seq":3167,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312876+0000","last_change":"2024-09-22T01:03:41.886773+0000","last_active":"2024-09-22T01:04:11.312876+0000","last_peered":"2024-09-22T01:04:11.312876+0000","last_clean":"2024-09-22T01:04:11.312876+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:04:11.312876+0000","last_undegraded":"2024-09-22T01:04:11.312876+0000","last_fullsized":"2024-09-22T01:04:11.312876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:39.279522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":390,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314477+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:04:11.314477+0000","last_peered":"2024-09-22T01:04:11.314477+0000","last_clean":"2024-09-22T01:04:11.314477+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:04:11.314477+0000","last_undegraded":"2024-09-22T01:04:11.314477+0000","last_fullsized":"2024-09-22T01:04:11.314477+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'392","reported_seq":2401,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312604+0000","last_change":"2024-09-22T01:03:41.882465+0000","last_active":"2024-09-22T01:04:11.312604+0000","last_peered":"2024-09-22T01:04:11.312604+0000","last_clean":"2024-09-22T01:04:11.312604+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:04:11.312604+0000","last_undegraded":"2024-09-22T01:04:11.312604+0000","last_fullsized":"2024-09-22T01:04:11.312604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:12.779986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314378+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:04:11.314378+0000","last_peered":"2024-09-22T01:04:11.314378+0000","last_clean":"2024-09-22T01:04:11.314378+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:04:11.314378+0000","last_undegraded":"2024-09-22T01:04:11.314378+0000","last_fullsized":"2024-09-22T01:04:11.314378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":526,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313661+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:04:11.313661+0000","last_peered":"2024-09-22T01:04:11.313661+0000","last_clean":"2024-09-22T01:04:11.313661+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:04:11.313661+0000","last_undegraded":"2024-09-22T01:04:11.313661+0000","last_fullsized":"2024-09-22T01:04:11.313661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'487","reported_seq":10794,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312452+0000","last_change":"2024-09-22T01:03:41.883318+0000","last_active":"2024-09-22T01:04:11.312452+0000","last_peered":"2024-09-22T01:04:11.312452+0000","last_clean":"2024-09-22T01:04:11.312452+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:04:11.312452+0000","last_undegraded":"2024-09-22T01:04:11.312452+0000","last_fullsized":"2024-09-22T01:04:11.312452+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:37.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'603","reported_seq":2590,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912974+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:19.230227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'801","reported_seq":16464,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315079+0000","last_change":"2024-09-22T01:03:41.893384+0000","last_active":"2024-09-22T01:04:11.315079+0000","last_peered":"2024-09-22T01:04:11.315079+0000","last_clean":"2024-09-22T01:04:11.315079+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:04:11.315079+0000","last_undegraded":"2024-09-22T01:04:11.315079+0000","last_fullsized":"2024-09-22T01:04:11.315079+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:18.880496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"175'701","reported_seq":2989,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312731+0000","last_change":"2024-09-22T01:03:41.889194+0000","last_active":"2024-09-22T01:04:11.312731+0000","last_peered":"2024-09-22T01:04:11.312731+0000","last_clean":"2024-09-22T01:04:11.312731+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:04:11.312731+0000","last_undegraded":"2024-09-22T01:04:11.312731+0000","last_fullsized":"2024-09-22T01:04:11.312731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:52.453227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'418","reported_seq":1591,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314308+0000","last_change":"2024-09-22T01:03:41.906335+0000","last_active":"2024-09-22T01:04:11.314308+0000","last_peered":"2024-09-22T01:04:11.314308+0000","last_clean":"2024-09-22T01:04:11.314308+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:04:11.314308+0000","last_undegraded":"2024-09-22T01:04:11.314308+0000","last_fullsized":"2024-09-22T01:04:11.314308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:26.787651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'759","reported_seq":6026,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313835+0000","last_change":"2024-09-22T01:03:41.999171+0000","last_active":"2024-09-22T01:04:11.313835+0000","last_peered":"2024-09-22T01:04:11.313835+0000","last_clean":"2024-09-22T01:04:11.313835+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:04:11.313835+0000","last_undegraded":"2024-09-22T01:04:11.313835+0000","last_fullsized":"2024-09-22T01:04:11.313835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:25.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'486","reported_seq":5220,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314145+0000","last_change":"2024-09-22T01:03:42.000884+0000","last_active":"2024-09-22T01:04:11.314145+0000","last_peered":"2024-09-22T01:04:11.314145+0000","last_clean":"2024-09-22T01:04:11.314145+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:04:11.314145+0000","last_undegraded":"2024-09-22T01:04:11.314145+0000","last_fullsized":"2024-09-22T01:04:11.314145+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:01.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"174'399","reported_seq":5235,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313998+0000","last_change":"2024-09-22T01:03:42.001967+0000","last_active":"2024-09-22T01:04:11.313998+0000","last_peered":"2024-09-22T01:04:11.313998+0000","last_clean":"2024-09-22T01:04:11.313998+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:04:11.313998+0000","last_undegraded":"2024-09-22T01:04:11.313998+0000","last_fullsized":"2024-09-22T01:04:11.313998+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:14.538603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'462","reported_seq":3412,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315076+0000","last_change":"2024-09-22T01:03:41.893276+0000","last_active":"2024-09-22T01:04:11.315076+0000","last_peered":"2024-09-22T01:04:11.315076+0000","last_clean":"2024-09-22T01:04:11.315076+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:04:11.315076+0000","last_undegraded":"2024-09-22T01:04:11.315076+0000","last_fullsized":"2024-09-22T01:04:11.315076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:32.260241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'680","reported_seq":3066,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313567+0000","last_change":"2024-09-22T01:03:41.997899+0000","last_active":"2024-09-22T01:04:11.313567+0000","last_peered":"2024-09-22T01:04:11.313567+0000","last_clean":"2024-09-22T01:04:11.313567+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:04:11.313567+0000","last_undegraded":"2024-09-22T01:04:11.313567+0000","last_fullsized":"2024-09-22T01:04:11.313567+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.873636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'445","reported_seq":3271,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312929+0000","last_change":"2024-09-22T01:03:41.888133+0000","last_active":"2024-09-22T01:04:11.312929+0000","last_peered":"2024-09-22T01:04:11.312929+0000","last_clean":"2024-09-22T01:04:11.312929+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:04:11.312929+0000","last_undegraded":"2024-09-22T01:04:11.312929+0000","last_fullsized":"2024-09-22T01:04:11.312929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:14.342554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'314","reported_seq":6900,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312720+0000","last_change":"2024-09-22T01:03:41.881304+0000","last_active":"2024-09-22T01:04:11.312720+0000","last_peered":"2024-09-22T01:04:11.312720+0000","last_clean":"2024-09-22T01:04:11.312720+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:04:11.312720+0000","last_undegraded":"2024-09-22T01:04:11.312720+0000","last_fullsized":"2024-09-22T01:04:11.312720+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.277966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19947,"ondisk_log_size":19947,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33548,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338292,"statfs":{"total":96636764160,"available":96602411008,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299427,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":2344,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":2400256,"data_stored":1158550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55752,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316088,"statfs":{"total":96636764160,"available":96579674112,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47432,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324408,"statfs":{"total":96636764160,"available":96588193792,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51604,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320236,"statfs":{"total":96636764160,"available":96583921664,"internally_reserved":0,"allocated":2416640,"data_stored":1164498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48008,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323832,"statfs":{"total":96636764160,"available":96587603968,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:04:16.399 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:04:16.703 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:16.703 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":200,"fsid":"7d08b428-4185-4f0f-a0fc-f7dc1fefdce9","created":"2024-09-22T01:00:18.015746+0000","modified":"2024-09-22T01:04:10.911226+0000","last_up_change":"2024-09-22T01:01:26.960327+0000","last_in_change":"2024-09-22T01:03:05.961739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:00:30.699559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:00:36.410605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-22T01:00:46.342370+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":50,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c0f94a8-3c63-442d-9dc1-50c1278929da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6825","nonce":2277223948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6827","nonce":2277223948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6831","nonce":2277223948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2277223948},{"type":"v1","addr":"172.21.15.2:6829","nonce":2277223948}]},"public_addr":"172.21.15.2:6825/2277223948","cluster_addr":"172.21.15.2:6827/2277223948","heartbeat_back_addr":"172.21.15.2:6831/2277223948","heartbeat_front_addr":"172.21.15.2:6829/2277223948","state":["exists","up"]},{"osd":1,"uuid":"2d2cf30e-e9bd-40ba-8293-193faa02ea69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6817","nonce":463340194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6819","nonce":463340194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6823","nonce":463340194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":463340194},{"type":"v1","addr":"172.21.15.2:6821","nonce":463340194}]},"public_addr":"172.21.15.2:6817/463340194","cluster_addr":"172.21.15.2:6819/463340194","heartbeat_back_addr":"172.21.15.2:6823/463340194","heartbeat_front_addr":"172.21.15.2:6821/463340194","state":["exists","up"]},{"osd":2,"uuid":"a3d3e330-9ea1-4a5c-811b-f285f9a3cff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6801","nonce":387643397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6803","nonce":387643397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6807","nonce":387643397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":387643397},{"type":"v1","addr":"172.21.15.2:6805","nonce":387643397}]},"public_addr":"172.21.15.2:6801/387643397","cluster_addr":"172.21.15.2:6803/387643397","heartbeat_back_addr":"172.21.15.2:6807/387643397","heartbeat_front_addr":"172.21.15.2:6805/387643397","state":["exists","up"]},{"osd":3,"uuid":"239072f0-4b69-4624-ba6c-733f299208da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6809","nonce":1215210764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6811","nonce":1215210764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6815","nonce":1215210764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1215210764},{"type":"v1","addr":"172.21.15.2:6813","nonce":1215210764}]},"public_addr":"172.21.15.2:6809/1215210764","cluster_addr":"172.21.15.2:6811/1215210764","heartbeat_back_addr":"172.21.15.2:6815/1215210764","heartbeat_front_addr":"172.21.15.2:6813/1215210764","state":["exists","up"]},{"osd":4,"uuid":"fd35cb24-eb2c-4c17-abe9-ce5c34db9014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6801","nonce":2463696152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6803","nonce":2463696152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6807","nonce":2463696152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2463696152},{"type":"v1","addr":"172.21.15.119:6805","nonce":2463696152}]},"public_addr":"172.21.15.119:6801/2463696152","cluster_addr":"172.21.15.119:6803/2463696152","heartbeat_back_addr":"172.21.15.119:6807/2463696152","heartbeat_front_addr":"172.21.15.119:6805/2463696152","state":["exists","up"]},{"osd":5,"uuid":"b60a584d-5334-41e4-92d9-f6689de6b8bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":25,"up_from":52,"up_thru":145,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6825","nonce":1737475131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6827","nonce":1737475131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6831","nonce":1737475131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1737475131},{"type":"v1","addr":"172.21.15.119:6829","nonce":1737475131}]},"public_addr":"172.21.15.119:6825/1737475131","cluster_addr":"172.21.15.119:6827/1737475131","heartbeat_back_addr":"172.21.15.119:6831/1737475131","heartbeat_front_addr":"172.21.15.119:6829/1737475131","state":["exists","up"]},{"osd":6,"uuid":"9be24894-1100-4e4e-bf1b-79e163fd97e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6817","nonce":4220217922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6819","nonce":4220217922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6823","nonce":4220217922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4220217922},{"type":"v1","addr":"172.21.15.119:6821","nonce":4220217922}]},"public_addr":"172.21.15.119:6817/4220217922","cluster_addr":"172.21.15.119:6819/4220217922","heartbeat_back_addr":"172.21.15.119:6823/4220217922","heartbeat_front_addr":"172.21.15.119:6821/4220217922","state":["exists","up"]},{"osd":7,"uuid":"293cfbc3-8c2e-40b5-ab89-0a124cc8651c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6809","nonce":3867156106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6811","nonce":3867156106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6815","nonce":3867156106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3867156106},{"type":"v1","addr":"172.21.15.119:6813","nonce":3867156106}]},"public_addr":"172.21.15.119:6809/3867156106","cluster_addr":"172.21.15.119:6811/3867156106","heartbeat_back_addr":"172.21.15.119:6815/3867156106","heartbeat_front_addr":"172.21.15.119:6813/3867156106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.556128+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.159753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.766199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.155529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.226232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.243346+0000","dead_epoch":26},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:28.143084+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:00:27.524369+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.17","osds":[6,1]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":1,"to":2}]},{"pgid":"3.a","mappings":[{"from":4,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:04:17.723 INFO:tasks.ceph:Scrubbing osd.0 2024-09-22T01:04:17.723 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:04:17.889 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:04:17.889 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:04:17.889 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:04:17.905 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-22T01:04:18.203 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 0 to deep-scrub 2024-09-22T01:04:18.222 INFO:tasks.ceph:Scrubbing osd.1 2024-09-22T01:04:18.223 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:04:18.363 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:04:18.363 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:04:18.363 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:04:18.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-22T01:04:18.675 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 1 to deep-scrub 2024-09-22T01:04:18.695 INFO:tasks.ceph:Scrubbing osd.2 2024-09-22T01:04:18.695 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:04:18.847 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:04:18.847 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:04:18.847 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:04:18.863 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-22T01:04:19.160 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 2 to deep-scrub 2024-09-22T01:04:19.179 INFO:tasks.ceph:Scrubbing osd.3 2024-09-22T01:04:19.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:04:19.321 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:04:19.321 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:04:19.321 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:04:19.336 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-22T01:04:19.625 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 3 to deep-scrub 2024-09-22T01:04:19.645 INFO:tasks.ceph:Scrubbing osd.4 2024-09-22T01:04:19.645 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:04:19.801 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:04:19.802 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:04:19.802 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:04:19.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-22T01:04:20.115 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 4 to deep-scrub 2024-09-22T01:04:20.134 INFO:tasks.ceph:Scrubbing osd.5 2024-09-22T01:04:20.134 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:04:20.285 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:04:20.285 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:04:20.285 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:04:20.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-22T01:04:20.597 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 5 to deep-scrub 2024-09-22T01:04:20.619 INFO:tasks.ceph:Scrubbing osd.6 2024-09-22T01:04:20.619 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:04:20.769 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:04:20.769 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:04:20.769 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:04:20.786 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-22T01:04:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 6 to deep-scrub 2024-09-22T01:04:21.104 INFO:tasks.ceph:Scrubbing osd.7 2024-09-22T01:04:21.104 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:04:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-22T01:04:21.447+0000 7f5091e6f640 0 --2- 172.21.15.2:0/2117940766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f508c072480 0x7f508c13bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-22T01:04:21.475 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-22T01:04:21.475 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:04:21.475 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-22T01:04:21.497 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-22T01:04:21.816 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 7 to deep-scrub 2024-09-22T01:04:21.836 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:04:22.142 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:22.142 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:04:22.161 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-22T01:04:20.678450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20067,"ondisk_log_size":20067,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378908,"kb_used_data":19992,"kb_used_omap":15,"kb_used_meta":358704,"kb_avail":754595812,"statfs":{"total":773094113280,"available":772706111488,"internally_reserved":0,"allocated":20471808,"data_stored":10496492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":367313593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003261"},"pg_stats":[{"pgid":"3.1f","version":"176'532","reported_seq":7206,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313143+0000","last_change":"2024-09-22T01:03:41.888720+0000","last_active":"2024-09-22T01:04:11.313143+0000","last_peered":"2024-09-22T01:04:11.313143+0000","last_clean":"2024-09-22T01:04:11.313143+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:04:11.313143+0000","last_undegraded":"2024-09-22T01:04:11.313143+0000","last_fullsized":"2024-09-22T01:04:11.313143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:31.136135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"176'811","reported_seq":3869,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312644+0000","last_change":"2024-09-22T01:03:41.885281+0000","last_active":"2024-09-22T01:04:11.312644+0000","last_peered":"2024-09-22T01:04:11.312644+0000","last_clean":"2024-09-22T01:04:11.312644+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:04:11.312644+0000","last_undegraded":"2024-09-22T01:04:11.312644+0000","last_fullsized":"2024-09-22T01:04:11.312644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:49.748988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'502","reported_seq":5361,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313981+0000","last_change":"2024-09-22T01:03:41.907575+0000","last_active":"2024-09-22T01:04:11.313981+0000","last_peered":"2024-09-22T01:04:11.313981+0000","last_clean":"2024-09-22T01:04:11.313981+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:04:11.313981+0000","last_undegraded":"2024-09-22T01:04:11.313981+0000","last_fullsized":"2024-09-22T01:04:11.313981+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:35:17.317230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1126","reported_seq":10745,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314011+0000","last_change":"2024-09-22T01:03:41.905727+0000","last_active":"2024-09-22T01:04:11.314011+0000","last_peered":"2024-09-22T01:04:11.314011+0000","last_clean":"2024-09-22T01:04:11.314011+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:04:11.314011+0000","last_undegraded":"2024-09-22T01:04:11.314011+0000","last_fullsized":"2024-09-22T01:04:11.314011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:36.367987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'1098","reported_seq":3143,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313103+0000","last_change":"2024-09-22T01:03:41.889650+0000","last_active":"2024-09-22T01:04:11.313103+0000","last_peered":"2024-09-22T01:04:11.313103+0000","last_clean":"2024-09-22T01:04:11.313103+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:04:11.313103+0000","last_undegraded":"2024-09-22T01:04:11.313103+0000","last_fullsized":"2024-09-22T01:04:11.313103+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:09.100865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'484","reported_seq":12400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912630+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:30:44.630235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'401","reported_seq":10093,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919216+0000","last_change":"2024-09-22T01:03:41.888321+0000","last_active":"2024-09-22T01:04:10.919216+0000","last_peered":"2024-09-22T01:04:10.919216+0000","last_clean":"2024-09-22T01:04:10.919216+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:04:10.919216+0000","last_undegraded":"2024-09-22T01:04:10.919216+0000","last_fullsized":"2024-09-22T01:04:10.919216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:18.365366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"176'793","reported_seq":6641,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312433+0000","last_change":"2024-09-22T01:03:41.882274+0000","last_active":"2024-09-22T01:04:11.312433+0000","last_peered":"2024-09-22T01:04:11.312433+0000","last_clean":"2024-09-22T01:04:11.312433+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:04:11.312433+0000","last_undegraded":"2024-09-22T01:04:11.312433+0000","last_fullsized":"2024-09-22T01:04:11.312433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:27:12.860843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'563","reported_seq":4411,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315369+0000","last_change":"2024-09-22T01:03:41.894428+0000","last_active":"2024-09-22T01:04:11.315369+0000","last_peered":"2024-09-22T01:04:11.315369+0000","last_clean":"2024-09-22T01:04:11.315369+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:04:11.315369+0000","last_undegraded":"2024-09-22T01:04:11.315369+0000","last_fullsized":"2024-09-22T01:04:11.315369+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:55.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"175'369","reported_seq":2285,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314632+0000","last_change":"2024-09-22T01:03:41.907362+0000","last_active":"2024-09-22T01:04:11.314632+0000","last_peered":"2024-09-22T01:04:11.314632+0000","last_clean":"2024-09-22T01:04:11.314632+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:04:11.314632+0000","last_undegraded":"2024-09-22T01:04:11.314632+0000","last_fullsized":"2024-09-22T01:04:11.314632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:08.940607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'687","reported_seq":6885,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313654+0000","last_change":"2024-09-22T01:03:41.999115+0000","last_active":"2024-09-22T01:04:11.313654+0000","last_peered":"2024-09-22T01:04:11.313654+0000","last_clean":"2024-09-22T01:04:11.313654+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:04:11.313654+0000","last_undegraded":"2024-09-22T01:04:11.313654+0000","last_fullsized":"2024-09-22T01:04:11.313654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:33.921063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'563","reported_seq":13986,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918873+0000","last_change":"2024-09-22T01:03:41.912104+0000","last_active":"2024-09-22T01:04:10.918873+0000","last_peered":"2024-09-22T01:04:10.918873+0000","last_clean":"2024-09-22T01:04:10.918873+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:04:10.918873+0000","last_undegraded":"2024-09-22T01:04:10.918873+0000","last_fullsized":"2024-09-22T01:04:10.918873+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:24:42.625118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":376,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315371+0000","last_change":"2024-09-22T01:00:40.193083+0000","last_active":"2024-09-22T01:04:11.315371+0000","last_peered":"2024-09-22T01:04:11.315371+0000","last_clean":"2024-09-22T01:04:11.315371+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:04:11.315371+0000","last_undegraded":"2024-09-22T01:04:11.315371+0000","last_fullsized":"2024-09-22T01:04:11.315371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:27.465638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"176'605","reported_seq":3321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313152+0000","last_change":"2024-09-22T01:03:41.887491+0000","last_active":"2024-09-22T01:04:11.313152+0000","last_peered":"2024-09-22T01:04:11.313152+0000","last_clean":"2024-09-22T01:04:11.313152+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:04:11.313152+0000","last_undegraded":"2024-09-22T01:04:11.313152+0000","last_fullsized":"2024-09-22T01:04:11.313152+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:31.037935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919602+0000","last_change":"2024-09-22T01:02:35.044723+0000","last_active":"2024-09-22T01:04:10.919602+0000","last_peered":"2024-09-22T01:04:10.919602+0000","last_clean":"2024-09-22T01:04:10.919602+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:04:10.919602+0000","last_undegraded":"2024-09-22T01:04:10.919602+0000","last_fullsized":"2024-09-22T01:04:10.919602+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:20.025054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'565","reported_seq":3653,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312775+0000","last_change":"2024-09-22T01:03:41.886088+0000","last_active":"2024-09-22T01:04:11.312775+0000","last_peered":"2024-09-22T01:04:11.312775+0000","last_clean":"2024-09-22T01:04:11.312775+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:04:11.312775+0000","last_undegraded":"2024-09-22T01:04:11.312775+0000","last_fullsized":"2024-09-22T01:04:11.312775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:49.543414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":375,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313917+0000","last_change":"2024-09-22T01:00:40.354336+0000","last_active":"2024-09-22T01:04:11.313917+0000","last_peered":"2024-09-22T01:04:11.313917+0000","last_clean":"2024-09-22T01:04:11.313917+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:04:11.313917+0000","last_undegraded":"2024-09-22T01:04:11.313917+0000","last_fullsized":"2024-09-22T01:04:11.313917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.872327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'444","reported_seq":4520,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919765+0000","last_change":"2024-09-22T01:03:41.883039+0000","last_active":"2024-09-22T01:04:10.919765+0000","last_peered":"2024-09-22T01:04:10.919765+0000","last_clean":"2024-09-22T01:04:10.919765+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:04:10.919765+0000","last_undegraded":"2024-09-22T01:04:10.919765+0000","last_fullsized":"2024-09-22T01:04:10.919765+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:52.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":192,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919513+0000","last_change":"2024-09-22T01:02:35.044920+0000","last_active":"2024-09-22T01:04:10.919513+0000","last_peered":"2024-09-22T01:04:10.919513+0000","last_clean":"2024-09-22T01:04:10.919513+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:04:10.919513+0000","last_undegraded":"2024-09-22T01:04:10.919513+0000","last_fullsized":"2024-09-22T01:04:10.919513+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:32.204422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'680","reported_seq":4687,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314274+0000","last_change":"2024-09-22T01:03:41.906860+0000","last_active":"2024-09-22T01:04:11.314274+0000","last_peered":"2024-09-22T01:04:11.314274+0000","last_clean":"2024-09-22T01:04:11.314274+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:04:11.314274+0000","last_undegraded":"2024-09-22T01:04:11.314274+0000","last_fullsized":"2024-09-22T01:04:11.314274+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:43.965472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"176'736","reported_seq":3097,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918646+0000","last_change":"2024-09-22T01:03:41.913492+0000","last_active":"2024-09-22T01:04:10.918646+0000","last_peered":"2024-09-22T01:04:10.918646+0000","last_clean":"2024-09-22T01:04:10.918646+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:04:10.918646+0000","last_undegraded":"2024-09-22T01:04:10.918646+0000","last_fullsized":"2024-09-22T01:04:10.918646+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:27.079591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":349,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314672+0000","last_change":"2024-09-22T01:02:35.052599+0000","last_active":"2024-09-22T01:04:11.314672+0000","last_peered":"2024-09-22T01:04:11.314672+0000","last_clean":"2024-09-22T01:04:11.314672+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:04:11.314672+0000","last_undegraded":"2024-09-22T01:04:11.314672+0000","last_fullsized":"2024-09-22T01:04:11.314672+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:07.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'1653","reported_seq":3951,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.919226+0000","last_change":"2024-09-22T01:03:41.883040+0000","last_active":"2024-09-22T01:04:10.919226+0000","last_peered":"2024-09-22T01:04:10.919226+0000","last_clean":"2024-09-22T01:04:10.919226+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:04:10.919226+0000","last_undegraded":"2024-09-22T01:04:10.919226+0000","last_fullsized":"2024-09-22T01:04:10.919226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":1653,"log_dups_size":0,"ondisk_log_size":1653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:43.607282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":377,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.916415+0000","last_change":"2024-09-22T01:00:39.945210+0000","last_active":"2024-09-22T01:04:10.916415+0000","last_peered":"2024-09-22T01:04:10.916415+0000","last_clean":"2024-09-22T01:04:10.916415+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:04:10.916415+0000","last_undegraded":"2024-09-22T01:04:10.916415+0000","last_fullsized":"2024-09-22T01:04:10.916415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:29.699864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'388","reported_seq":3167,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312876+0000","last_change":"2024-09-22T01:03:41.886773+0000","last_active":"2024-09-22T01:04:11.312876+0000","last_peered":"2024-09-22T01:04:11.312876+0000","last_clean":"2024-09-22T01:04:11.312876+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:04:11.312876+0000","last_undegraded":"2024-09-22T01:04:11.312876+0000","last_fullsized":"2024-09-22T01:04:11.312876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:39.279522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":390,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314477+0000","last_change":"2024-09-22T01:03:13.744659+0000","last_active":"2024-09-22T01:04:11.314477+0000","last_peered":"2024-09-22T01:04:11.314477+0000","last_clean":"2024-09-22T01:04:11.314477+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:04:11.314477+0000","last_undegraded":"2024-09-22T01:04:11.314477+0000","last_fullsized":"2024-09-22T01:04:11.314477+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:25.466663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'392","reported_seq":2401,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312604+0000","last_change":"2024-09-22T01:03:41.882465+0000","last_active":"2024-09-22T01:04:11.312604+0000","last_peered":"2024-09-22T01:04:11.312604+0000","last_clean":"2024-09-22T01:04:11.312604+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:04:11.312604+0000","last_undegraded":"2024-09-22T01:04:11.312604+0000","last_fullsized":"2024-09-22T01:04:11.312604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:12.779986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314378+0000","last_change":"2024-09-22T01:01:27.722449+0000","last_active":"2024-09-22T01:04:11.314378+0000","last_peered":"2024-09-22T01:04:11.314378+0000","last_clean":"2024-09-22T01:04:11.314378+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:04:11.314378+0000","last_undegraded":"2024-09-22T01:04:11.314378+0000","last_fullsized":"2024-09-22T01:04:11.314378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:36.902696+0000","last_clean_scrub_stamp":"2024-09-22T01:00:36.902696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:50.444538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":526,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313661+0000","last_change":"2024-09-22T01:00:32.500194+0000","last_active":"2024-09-22T01:04:11.313661+0000","last_peered":"2024-09-22T01:04:11.313661+0000","last_clean":"2024-09-22T01:04:11.313661+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:04:11.313661+0000","last_undegraded":"2024-09-22T01:04:11.313661+0000","last_fullsized":"2024-09-22T01:04:11.313661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:30.876018+0000","last_clean_scrub_stamp":"2024-09-22T01:00:30.876018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:01.198733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'487","reported_seq":10794,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312452+0000","last_change":"2024-09-22T01:03:41.883318+0000","last_active":"2024-09-22T01:04:11.312452+0000","last_peered":"2024-09-22T01:04:11.312452+0000","last_clean":"2024-09-22T01:04:11.312452+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:04:11.312452+0000","last_undegraded":"2024-09-22T01:04:11.312452+0000","last_fullsized":"2024-09-22T01:04:11.312452+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:37.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'603","reported_seq":2590,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:10.918457+0000","last_change":"2024-09-22T01:03:41.912974+0000","last_active":"2024-09-22T01:04:10.918457+0000","last_peered":"2024-09-22T01:04:10.918457+0000","last_clean":"2024-09-22T01:04:10.918457+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:04:10.918457+0000","last_undegraded":"2024-09-22T01:04:10.918457+0000","last_fullsized":"2024-09-22T01:04:10.918457+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:51:19.230227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'801","reported_seq":16464,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315079+0000","last_change":"2024-09-22T01:03:41.893384+0000","last_active":"2024-09-22T01:04:11.315079+0000","last_peered":"2024-09-22T01:04:11.315079+0000","last_clean":"2024-09-22T01:04:11.315079+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:04:11.315079+0000","last_undegraded":"2024-09-22T01:04:11.315079+0000","last_fullsized":"2024-09-22T01:04:11.315079+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:18.880496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"175'701","reported_seq":2989,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312731+0000","last_change":"2024-09-22T01:03:41.889194+0000","last_active":"2024-09-22T01:04:11.312731+0000","last_peered":"2024-09-22T01:04:11.312731+0000","last_clean":"2024-09-22T01:04:11.312731+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:04:11.312731+0000","last_undegraded":"2024-09-22T01:04:11.312731+0000","last_fullsized":"2024-09-22T01:04:11.312731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:52.453227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'418","reported_seq":1591,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314308+0000","last_change":"2024-09-22T01:03:41.906335+0000","last_active":"2024-09-22T01:04:11.314308+0000","last_peered":"2024-09-22T01:04:11.314308+0000","last_clean":"2024-09-22T01:04:11.314308+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:04:11.314308+0000","last_undegraded":"2024-09-22T01:04:11.314308+0000","last_fullsized":"2024-09-22T01:04:11.314308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:26.787651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'759","reported_seq":6026,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313835+0000","last_change":"2024-09-22T01:03:41.999171+0000","last_active":"2024-09-22T01:04:11.313835+0000","last_peered":"2024-09-22T01:04:11.313835+0000","last_clean":"2024-09-22T01:04:11.313835+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:04:11.313835+0000","last_undegraded":"2024-09-22T01:04:11.313835+0000","last_fullsized":"2024-09-22T01:04:11.313835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:25.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'486","reported_seq":5220,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.314145+0000","last_change":"2024-09-22T01:03:42.000884+0000","last_active":"2024-09-22T01:04:11.314145+0000","last_peered":"2024-09-22T01:04:11.314145+0000","last_clean":"2024-09-22T01:04:11.314145+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:04:11.314145+0000","last_undegraded":"2024-09-22T01:04:11.314145+0000","last_fullsized":"2024-09-22T01:04:11.314145+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:34:01.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"174'399","reported_seq":5235,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313998+0000","last_change":"2024-09-22T01:03:42.001967+0000","last_active":"2024-09-22T01:04:11.313998+0000","last_peered":"2024-09-22T01:04:11.313998+0000","last_clean":"2024-09-22T01:04:11.313998+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:04:11.313998+0000","last_undegraded":"2024-09-22T01:04:11.313998+0000","last_fullsized":"2024-09-22T01:04:11.313998+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:14.538603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'462","reported_seq":3412,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.315076+0000","last_change":"2024-09-22T01:03:41.893276+0000","last_active":"2024-09-22T01:04:11.315076+0000","last_peered":"2024-09-22T01:04:11.315076+0000","last_clean":"2024-09-22T01:04:11.315076+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:04:11.315076+0000","last_undegraded":"2024-09-22T01:04:11.315076+0000","last_fullsized":"2024-09-22T01:04:11.315076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:32.260241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'680","reported_seq":3066,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.313567+0000","last_change":"2024-09-22T01:03:41.997899+0000","last_active":"2024-09-22T01:04:11.313567+0000","last_peered":"2024-09-22T01:04:11.313567+0000","last_clean":"2024-09-22T01:04:11.313567+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:04:11.313567+0000","last_undegraded":"2024-09-22T01:04:11.313567+0000","last_fullsized":"2024-09-22T01:04:11.313567+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.873636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'445","reported_seq":3271,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312929+0000","last_change":"2024-09-22T01:03:41.888133+0000","last_active":"2024-09-22T01:04:11.312929+0000","last_peered":"2024-09-22T01:04:11.312929+0000","last_clean":"2024-09-22T01:04:11.312929+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:04:11.312929+0000","last_undegraded":"2024-09-22T01:04:11.312929+0000","last_fullsized":"2024-09-22T01:04:11.312929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:14.342554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'314","reported_seq":6900,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:11.312720+0000","last_change":"2024-09-22T01:03:41.881304+0000","last_active":"2024-09-22T01:04:11.312720+0000","last_peered":"2024-09-22T01:04:11.312720+0000","last_clean":"2024-09-22T01:04:11.312720+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:04:11.312720+0000","last_undegraded":"2024-09-22T01:04:11.312720+0000","last_fullsized":"2024-09-22T01:04:11.312720+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:00:46.960642+0000","last_clean_scrub_stamp":"2024-09-22T01:00:46.960642+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.277966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":19947,"ondisk_log_size":19947,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33548,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338292,"statfs":{"total":96636764160,"available":96602411008,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299428,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30484,"kb_used_data":2356,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341356,"statfs":{"total":96636764160,"available":96605548544,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542194,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542194,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55752,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316088,"statfs":{"total":96636764160,"available":96579674112,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47432,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324408,"statfs":{"total":96636764160,"available":96588193792,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542193,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51604,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320236,"statfs":{"total":96636764160,"available":96583921664,"internally_reserved":0,"allocated":2416640,"data_stored":1164498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48008,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323832,"statfs":{"total":96636764160,"available":96587603968,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:04:22.166 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.166 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.166 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.166 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.166 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-22T01:00:36.902696+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=36, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.167 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-22T01:00:36.902696+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=36, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-22T01:00:36.902696+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=36, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-22T01:00:36.902696+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=36, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-22T01:00:36.902696+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=36, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.168 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-22T01:00:36.902696+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=36, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.169 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.169 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-22T01:00:36.902696+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=36, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.169 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.169 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-22T01:00:36.902696+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=36, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.169 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-22T01:00:30.876018+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=30, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.169 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.169 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.170 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.170 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.170 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.170 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.170 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.170 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.170 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.170 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.171 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.171 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-22T01:00:46.960642+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=0, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=4, tm_sec=16, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:04:22.171 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-22T01:04:42.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:04:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-22T01:04:42.495 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-22T01:04:42.511 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-22T01:04:40.684944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":256,"num_read":203373,"num_read_kb":760566,"num_write":22786,"num_write_kb":413526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":211294816,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20331,"ondisk_log_size":20331,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":383000,"kb_used_data":19992,"kb_used_omap":15,"kb_used_meta":362864,"kb_avail":754591720,"statfs":{"total":773094113280,"available":772701921280,"internally_reserved":0,"allocated":20471808,"data_stored":10496492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15687,"internal_metadata":371573433},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003632"},"pg_stats":[{"pgid":"3.1f","version":"200'537","reported_seq":7220,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:19.571990+0000","last_change":"2024-09-22T01:04:19.572068+0000","last_active":"2024-09-22T01:04:19.571990+0000","last_peered":"2024-09-22T01:04:19.571990+0000","last_clean":"2024-09-22T01:04:19.571990+0000","last_became_active":"2024-09-22T01:01:31.020610+0000","last_became_peered":"2024-09-22T01:01:31.020610+0000","last_unstale":"2024-09-22T01:04:19.571990+0000","last_undegraded":"2024-09-22T01:04:19.571990+0000","last_fullsized":"2024-09-22T01:04:19.571990+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'537","last_scrub_stamp":"2024-09-22T01:04:19.571796+0000","last_deep_scrub":"200'537","last_deep_scrub_stamp":"2024-09-22T01:04:19.571796+0000","last_clean_scrub_stamp":"2024-09-22T01:04:19.571796+0000","objects_scrubbed":5,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:54.232508+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0037175160000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6538,"num_read_kb":45964,"num_write":554,"num_write_kb":17798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":4908190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"200'820","reported_seq":3887,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:23.529784+0000","last_change":"2024-09-22T01:04:23.529980+0000","last_active":"2024-09-22T01:04:23.529784+0000","last_peered":"2024-09-22T01:04:23.529784+0000","last_clean":"2024-09-22T01:04:23.529784+0000","last_became_active":"2024-09-22T01:00:47.725749+0000","last_became_peered":"2024-09-22T01:00:47.725749+0000","last_unstale":"2024-09-22T01:04:23.529784+0000","last_undegraded":"2024-09-22T01:04:23.529784+0000","last_fullsized":"2024-09-22T01:04:23.529784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'820","last_scrub_stamp":"2024-09-22T01:04:23.529524+0000","last_deep_scrub":"200'820","last_deep_scrub_stamp":"2024-09-22T01:04:23.529524+0000","last_clean_scrub_stamp":"2024-09-22T01:04:23.529524+0000","objects_scrubbed":9,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:38.394941+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0059781460000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6673,"num_read_kb":13262,"num_write":1007,"num_write_kb":7214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"200'515","reported_seq":5383,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:26.094307+0000","last_change":"2024-09-22T01:04:26.094386+0000","last_active":"2024-09-22T01:04:26.094307+0000","last_peered":"2024-09-22T01:04:26.094307+0000","last_clean":"2024-09-22T01:04:26.094307+0000","last_became_active":"2024-09-22T01:03:09.754133+0000","last_became_peered":"2024-09-22T01:03:09.754133+0000","last_unstale":"2024-09-22T01:04:26.094307+0000","last_undegraded":"2024-09-22T01:04:26.094307+0000","last_fullsized":"2024-09-22T01:04:26.094307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'515","last_scrub_stamp":"2024-09-22T01:04:26.094113+0000","last_deep_scrub":"200'515","last_deep_scrub_stamp":"2024-09-22T01:04:26.094113+0000","last_clean_scrub_stamp":"2024-09-22T01:04:26.094113+0000","objects_scrubbed":13,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:46:07.770704+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0029999620000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5110,"num_read_kb":32604,"num_write":505,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":11768218,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"200'1133","reported_seq":10761,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:27.061106+0000","last_change":"2024-09-22T01:04:27.061191+0000","last_active":"2024-09-22T01:04:27.061106+0000","last_peered":"2024-09-22T01:04:27.061106+0000","last_clean":"2024-09-22T01:04:27.061106+0000","last_became_active":"2024-09-22T01:03:11.746999+0000","last_became_peered":"2024-09-22T01:03:11.746999+0000","last_unstale":"2024-09-22T01:04:27.061106+0000","last_undegraded":"2024-09-22T01:04:27.061106+0000","last_fullsized":"2024-09-22T01:04:27.061106+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'1133","last_scrub_stamp":"2024-09-22T01:04:27.060919+0000","last_deep_scrub":"200'1133","last_deep_scrub_stamp":"2024-09-22T01:04:27.060919+0000","last_clean_scrub_stamp":"2024-09-22T01:04:27.060919+0000","objects_scrubbed":7,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:58:26.254915+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.001253964,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":11540,"num_read_kb":51761,"num_write":1615,"num_write_kb":18279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":36977570,"num_keys_recovered":326,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"200'1101","reported_seq":3155,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:22.635747+0000","last_change":"2024-09-22T01:04:22.635812+0000","last_active":"2024-09-22T01:04:22.635747+0000","last_peered":"2024-09-22T01:04:22.635747+0000","last_clean":"2024-09-22T01:04:22.635747+0000","last_became_active":"2024-09-22T01:03:12.750096+0000","last_became_peered":"2024-09-22T01:03:12.750096+0000","last_unstale":"2024-09-22T01:04:22.635747+0000","last_undegraded":"2024-09-22T01:04:22.635747+0000","last_fullsized":"2024-09-22T01:04:22.635747+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'1101","last_scrub_stamp":"2024-09-22T01:04:22.635527+0000","last_deep_scrub":"200'1101","last_deep_scrub_stamp":"2024-09-22T01:04:22.635527+0000","last_clean_scrub_stamp":"2024-09-22T01:04:22.635527+0000","objects_scrubbed":3,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:25:25.317269+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0040020769999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5211,"num_read_kb":15422,"num_write":1522,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":210648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"200'492","reported_seq":12417,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:20.682690+0000","last_change":"2024-09-22T01:04:20.682767+0000","last_active":"2024-09-22T01:04:20.682690+0000","last_peered":"2024-09-22T01:04:20.682690+0000","last_clean":"2024-09-22T01:04:20.682690+0000","last_became_active":"2024-09-22T01:03:09.750340+0000","last_became_peered":"2024-09-22T01:03:09.750340+0000","last_unstale":"2024-09-22T01:04:20.682690+0000","last_undegraded":"2024-09-22T01:04:20.682690+0000","last_fullsized":"2024-09-22T01:04:20.682690+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'492","last_scrub_stamp":"2024-09-22T01:04:20.682507+0000","last_deep_scrub":"200'492","last_deep_scrub_stamp":"2024-09-22T01:04:20.682507+0000","last_clean_scrub_stamp":"2024-09-22T01:04:20.682507+0000","objects_scrubbed":8,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:48:20.664451+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00167947,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12179,"num_read_kb":26019,"num_write":542,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":8130196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"200'407","reported_seq":10108,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:20.731051+0000","last_change":"2024-09-22T01:04:20.731103+0000","last_active":"2024-09-22T01:04:20.731051+0000","last_peered":"2024-09-22T01:04:20.731051+0000","last_clean":"2024-09-22T01:04:20.731051+0000","last_became_active":"2024-09-22T01:02:37.736697+0000","last_became_peered":"2024-09-22T01:02:37.736697+0000","last_unstale":"2024-09-22T01:04:20.731051+0000","last_undegraded":"2024-09-22T01:04:20.731051+0000","last_fullsized":"2024-09-22T01:04:20.731051+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'407","last_scrub_stamp":"2024-09-22T01:04:20.730949+0000","last_deep_scrub":"200'407","last_deep_scrub_stamp":"2024-09-22T01:04:20.730949+0000","last_clean_scrub_stamp":"2024-09-22T01:04:20.730949+0000","objects_scrubbed":6,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:18.561636+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0074470079999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9135,"num_read_kb":31303,"num_write":456,"num_write_kb":9308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":7387228,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"200'796","reported_seq":6653,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:21.557044+0000","last_change":"2024-09-22T01:04:21.557122+0000","last_active":"2024-09-22T01:04:21.557044+0000","last_peered":"2024-09-22T01:04:21.557044+0000","last_clean":"2024-09-22T01:04:21.557044+0000","last_became_active":"2024-09-22T01:00:47.717872+0000","last_became_peered":"2024-09-22T01:00:47.717872+0000","last_unstale":"2024-09-22T01:04:21.557044+0000","last_undegraded":"2024-09-22T01:04:21.557044+0000","last_fullsized":"2024-09-22T01:04:21.557044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'796","last_scrub_stamp":"2024-09-22T01:04:21.556895+0000","last_deep_scrub":"200'796","last_deep_scrub_stamp":"2024-09-22T01:04:21.556895+0000","last_clean_scrub_stamp":"2024-09-22T01:04:21.556895+0000","objects_scrubbed":3,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:58:19.871651+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0038715329999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8611,"num_read_kb":42722,"num_write":853,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"200'569","reported_seq":4426,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:23.743315+0000","last_change":"2024-09-22T01:04:23.743409+0000","last_active":"2024-09-22T01:04:23.743315+0000","last_peered":"2024-09-22T01:04:23.743315+0000","last_clean":"2024-09-22T01:04:23.743315+0000","last_became_active":"2024-09-22T01:02:51.867831+0000","last_became_peered":"2024-09-22T01:02:51.867831+0000","last_unstale":"2024-09-22T01:04:23.743315+0000","last_undegraded":"2024-09-22T01:04:23.743315+0000","last_fullsized":"2024-09-22T01:04:23.743315+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'569","last_scrub_stamp":"2024-09-22T01:04:23.743130+0000","last_deep_scrub":"200'569","last_deep_scrub_stamp":"2024-09-22T01:04:23.743130+0000","last_clean_scrub_stamp":"2024-09-22T01:04:23.743130+0000","objects_scrubbed":6,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:30:15.345405+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0026440309999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4907,"num_read_kb":18393,"num_write":626,"num_write_kb":8522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7525651,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"200'376","reported_seq":2301,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:22.028246+0000","last_change":"2024-09-22T01:04:22.028385+0000","last_active":"2024-09-22T01:04:22.028246+0000","last_peered":"2024-09-22T01:04:22.028246+0000","last_clean":"2024-09-22T01:04:22.028246+0000","last_became_active":"2024-09-22T01:01:27.724487+0000","last_became_peered":"2024-09-22T01:01:27.724487+0000","last_unstale":"2024-09-22T01:04:22.028246+0000","last_undegraded":"2024-09-22T01:04:22.028246+0000","last_fullsized":"2024-09-22T01:04:22.028246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'376","last_scrub_stamp":"2024-09-22T01:04:22.028024+0000","last_deep_scrub":"200'376","last_deep_scrub_stamp":"2024-09-22T01:04:22.028024+0000","last_clean_scrub_stamp":"2024-09-22T01:04:22.028024+0000","objects_scrubbed":7,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:42:50.265806+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0020669669999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2593,"num_read_kb":34291,"num_write":391,"num_write_kb":28823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3768550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"200'697","reported_seq":6904,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:24.278984+0000","last_change":"2024-09-22T01:04:24.279064+0000","last_active":"2024-09-22T01:04:24.278984+0000","last_peered":"2024-09-22T01:04:24.278984+0000","last_clean":"2024-09-22T01:04:24.278984+0000","last_became_active":"2024-09-22T01:00:47.724484+0000","last_became_peered":"2024-09-22T01:00:47.724484+0000","last_unstale":"2024-09-22T01:04:24.278984+0000","last_undegraded":"2024-09-22T01:04:24.278984+0000","last_fullsized":"2024-09-22T01:04:24.278984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'697","last_scrub_stamp":"2024-09-22T01:04:24.278790+0000","last_deep_scrub":"200'697","last_deep_scrub_stamp":"2024-09-22T01:04:24.278790+0000","last_clean_scrub_stamp":"2024-09-22T01:04:24.278790+0000","objects_scrubbed":10,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:59.976577+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001855797,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8896,"num_read_kb":27135,"num_write":699,"num_write_kb":5766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"200'567","reported_seq":13999,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:22.694889+0000","last_change":"2024-09-22T01:04:22.695016+0000","last_active":"2024-09-22T01:04:22.694889+0000","last_peered":"2024-09-22T01:04:22.694889+0000","last_clean":"2024-09-22T01:04:22.694889+0000","last_became_active":"2024-09-22T01:03:11.747347+0000","last_became_peered":"2024-09-22T01:03:11.747347+0000","last_unstale":"2024-09-22T01:04:22.694889+0000","last_undegraded":"2024-09-22T01:04:22.694889+0000","last_fullsized":"2024-09-22T01:04:22.694889+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'567","last_scrub_stamp":"2024-09-22T01:04:22.694620+0000","last_deep_scrub":"200'567","last_deep_scrub_stamp":"2024-09-22T01:04:22.694620+0000","last_clean_scrub_stamp":"2024-09-22T01:04:22.694620+0000","objects_scrubbed":4,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:09.003385+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0011541640000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13909,"num_read_kb":40987,"num_write":568,"num_write_kb":17935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3821530,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":385,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:21.776672+0000","last_change":"2024-09-22T01:04:21.776727+0000","last_active":"2024-09-22T01:04:21.776672+0000","last_peered":"2024-09-22T01:04:21.776672+0000","last_clean":"2024-09-22T01:04:21.776672+0000","last_became_active":"2024-09-22T01:00:37.930505+0000","last_became_peered":"2024-09-22T01:00:37.930505+0000","last_unstale":"2024-09-22T01:04:21.776672+0000","last_undegraded":"2024-09-22T01:04:21.776672+0000","last_fullsized":"2024-09-22T01:04:21.776672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:04:21.776469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:04:21.776469+0000","last_clean_scrub_stamp":"2024-09-22T01:04:21.776469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:27.970368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071759000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"200'616","reported_seq":3341,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:20.605409+0000","last_change":"2024-09-22T01:04:20.605454+0000","last_active":"2024-09-22T01:04:20.605409+0000","last_peered":"2024-09-22T01:04:20.605409+0000","last_clean":"2024-09-22T01:04:20.605409+0000","last_became_active":"2024-09-22T01:02:37.737028+0000","last_became_peered":"2024-09-22T01:02:37.737028+0000","last_unstale":"2024-09-22T01:04:20.605409+0000","last_undegraded":"2024-09-22T01:04:20.605409+0000","last_fullsized":"2024-09-22T01:04:20.605409+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'616","last_scrub_stamp":"2024-09-22T01:04:20.605347+0000","last_deep_scrub":"200'616","last_deep_scrub_stamp":"2024-09-22T01:04:20.605347+0000","last_clean_scrub_stamp":"2024-09-22T01:04:20.605347+0000","objects_scrubbed":11,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:24.962655+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0026262799999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4331,"num_read_kb":6723,"num_write":558,"num_write_kb":3331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":2227811,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":201,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:18.729234+0000","last_change":"2024-09-22T01:04:18.729274+0000","last_active":"2024-09-22T01:04:18.729234+0000","last_peered":"2024-09-22T01:04:18.729234+0000","last_clean":"2024-09-22T01:04:18.729234+0000","last_became_active":"2024-09-22T01:02:35.043553+0000","last_became_peered":"2024-09-22T01:02:35.043553+0000","last_unstale":"2024-09-22T01:04:18.729234+0000","last_undegraded":"2024-09-22T01:04:18.729234+0000","last_fullsized":"2024-09-22T01:04:18.729234+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:04:18.729120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:04:18.729120+0000","last_clean_scrub_stamp":"2024-09-22T01:04:18.729120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:44.798651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"200'574","reported_seq":3671,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:22.559884+0000","last_change":"2024-09-22T01:04:22.559972+0000","last_active":"2024-09-22T01:04:22.559884+0000","last_peered":"2024-09-22T01:04:22.559884+0000","last_clean":"2024-09-22T01:04:22.559884+0000","last_became_active":"2024-09-22T01:00:47.725480+0000","last_became_peered":"2024-09-22T01:00:47.725480+0000","last_unstale":"2024-09-22T01:04:22.559884+0000","last_undegraded":"2024-09-22T01:04:22.559884+0000","last_fullsized":"2024-09-22T01:04:22.559884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'574","last_scrub_stamp":"2024-09-22T01:04:22.559671+0000","last_deep_scrub":"200'574","last_deep_scrub_stamp":"2024-09-22T01:04:22.559671+0000","last_clean_scrub_stamp":"2024-09-22T01:04:22.559671+0000","objects_scrubbed":9,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:18:36.490033+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0039984060000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4486,"num_read_kb":12711,"num_write":687,"num_write_kb":6973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":384,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:23.243007+0000","last_change":"2024-09-22T01:04:23.243060+0000","last_active":"2024-09-22T01:04:23.243007+0000","last_peered":"2024-09-22T01:04:23.243007+0000","last_clean":"2024-09-22T01:04:23.243007+0000","last_became_active":"2024-09-22T01:00:37.930120+0000","last_became_peered":"2024-09-22T01:00:37.930120+0000","last_unstale":"2024-09-22T01:04:23.243007+0000","last_undegraded":"2024-09-22T01:04:23.243007+0000","last_fullsized":"2024-09-22T01:04:23.243007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:04:23.242809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:04:23.242809+0000","last_clean_scrub_stamp":"2024-09-22T01:04:23.242809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:06:48.394776+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00077130799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"200'455","reported_seq":4540,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:21.744016+0000","last_change":"2024-09-22T01:04:21.744082+0000","last_active":"2024-09-22T01:04:21.744016+0000","last_peered":"2024-09-22T01:04:21.744016+0000","last_clean":"2024-09-22T01:04:21.744016+0000","last_became_active":"2024-09-22T01:02:38.736204+0000","last_became_peered":"2024-09-22T01:02:38.736204+0000","last_unstale":"2024-09-22T01:04:21.744016+0000","last_undegraded":"2024-09-22T01:04:21.744016+0000","last_fullsized":"2024-09-22T01:04:21.744016+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'455","last_scrub_stamp":"2024-09-22T01:04:21.743892+0000","last_deep_scrub":"200'455","last_deep_scrub_stamp":"2024-09-22T01:04:21.743892+0000","last_clean_scrub_stamp":"2024-09-22T01:04:21.743892+0000","objects_scrubbed":11,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:11:06.910624+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0033187910000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4002,"num_read_kb":26479,"num_write":412,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":17638513,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":201,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:19.721654+0000","last_change":"2024-09-22T01:04:19.721689+0000","last_active":"2024-09-22T01:04:19.721654+0000","last_peered":"2024-09-22T01:04:19.721654+0000","last_clean":"2024-09-22T01:04:19.721654+0000","last_became_active":"2024-09-22T01:02:35.044581+0000","last_became_peered":"2024-09-22T01:02:35.044581+0000","last_unstale":"2024-09-22T01:04:19.721654+0000","last_undegraded":"2024-09-22T01:04:19.721654+0000","last_fullsized":"2024-09-22T01:04:19.721654+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:04:19.721572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:04:19.721572+0000","last_clean_scrub_stamp":"2024-09-22T01:04:19.721572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:35.299867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"200'686","reported_seq":4702,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:24.073110+0000","last_change":"2024-09-22T01:04:24.073193+0000","last_active":"2024-09-22T01:04:24.073110+0000","last_peered":"2024-09-22T01:04:24.073110+0000","last_clean":"2024-09-22T01:04:24.073110+0000","last_became_active":"2024-09-22T01:01:31.018371+0000","last_became_peered":"2024-09-22T01:01:31.018371+0000","last_unstale":"2024-09-22T01:04:24.073110+0000","last_undegraded":"2024-09-22T01:04:24.073110+0000","last_fullsized":"2024-09-22T01:04:24.073110+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'686","last_scrub_stamp":"2024-09-22T01:04:24.072921+0000","last_deep_scrub":"200'686","last_deep_scrub_stamp":"2024-09-22T01:04:24.072921+0000","last_clean_scrub_stamp":"2024-09-22T01:04:24.072921+0000","objects_scrubbed":6,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:17:23.098376+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0021421209999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7219,"num_read_kb":12928,"num_write":588,"num_write_kb":5653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":14908,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"200'746","reported_seq":3116,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:23.692399+0000","last_change":"2024-09-22T01:04:23.692572+0000","last_active":"2024-09-22T01:04:23.692399+0000","last_peered":"2024-09-22T01:04:23.692399+0000","last_clean":"2024-09-22T01:04:23.692399+0000","last_became_active":"2024-09-22T01:03:12.750037+0000","last_became_peered":"2024-09-22T01:03:12.750037+0000","last_unstale":"2024-09-22T01:04:23.692399+0000","last_undegraded":"2024-09-22T01:04:23.692399+0000","last_fullsized":"2024-09-22T01:04:23.692399+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'746","last_scrub_stamp":"2024-09-22T01:04:23.692138+0000","last_deep_scrub":"200'746","last_deep_scrub_stamp":"2024-09-22T01:04:23.692138+0000","last_clean_scrub_stamp":"2024-09-22T01:04:23.692138+0000","objects_scrubbed":10,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:02.733606+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0021023539999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3411,"num_read_kb":18018,"num_write":755,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":6403188,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"200'3","reported_seq":359,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:25.028095+0000","last_change":"2024-09-22T01:04:25.028163+0000","last_active":"2024-09-22T01:04:25.028095+0000","last_peered":"2024-09-22T01:04:25.028095+0000","last_clean":"2024-09-22T01:04:25.028095+0000","last_became_active":"2024-09-22T01:02:35.039120+0000","last_became_peered":"2024-09-22T01:02:35.039120+0000","last_unstale":"2024-09-22T01:04:25.028095+0000","last_undegraded":"2024-09-22T01:04:25.028095+0000","last_fullsized":"2024-09-22T01:04:25.028095+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'3","last_scrub_stamp":"2024-09-22T01:04:25.027908+0000","last_deep_scrub":"200'3","last_deep_scrub_stamp":"2024-09-22T01:04:25.027908+0000","last_clean_scrub_stamp":"2024-09-22T01:04:25.027908+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:22.467299+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00081630500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"200'1659","reported_seq":3966,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:22.731107+0000","last_change":"2024-09-22T01:04:22.731197+0000","last_active":"2024-09-22T01:04:22.731107+0000","last_peered":"2024-09-22T01:04:22.731107+0000","last_clean":"2024-09-22T01:04:22.731107+0000","last_became_active":"2024-09-22T01:02:39.737916+0000","last_became_peered":"2024-09-22T01:02:39.737916+0000","last_unstale":"2024-09-22T01:04:22.731107+0000","last_undegraded":"2024-09-22T01:04:22.731107+0000","last_fullsized":"2024-09-22T01:04:22.731107+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'1659","last_scrub_stamp":"2024-09-22T01:04:22.730898+0000","last_deep_scrub":"200'1659","last_deep_scrub_stamp":"2024-09-22T01:04:22.730898+0000","last_clean_scrub_stamp":"2024-09-22T01:04:22.730898+0000","objects_scrubbed":6,"log_size":1659,"log_dups_size":0,"ondisk_log_size":1659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:33:45.042420+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0028109960000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":9106,"num_read_kb":25481,"num_write":2701,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":14088535,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":386,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:19.875889+0000","last_change":"2024-09-22T01:04:19.875912+0000","last_active":"2024-09-22T01:04:19.875889+0000","last_peered":"2024-09-22T01:04:19.875889+0000","last_clean":"2024-09-22T01:04:19.875889+0000","last_became_active":"2024-09-22T01:00:37.959280+0000","last_became_peered":"2024-09-22T01:00:37.959280+0000","last_unstale":"2024-09-22T01:04:19.875889+0000","last_undegraded":"2024-09-22T01:04:19.875889+0000","last_fullsized":"2024-09-22T01:04:19.875889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:04:19.875806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:04:19.875806+0000","last_clean_scrub_stamp":"2024-09-22T01:04:19.875806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:21:27.771466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011510909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"200'397","reported_seq":3185,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:21.652078+0000","last_change":"2024-09-22T01:04:21.652112+0000","last_active":"2024-09-22T01:04:21.652078+0000","last_peered":"2024-09-22T01:04:21.652078+0000","last_clean":"2024-09-22T01:04:21.652078+0000","last_became_active":"2024-09-22T01:03:10.746909+0000","last_became_peered":"2024-09-22T01:03:10.746909+0000","last_unstale":"2024-09-22T01:04:21.652078+0000","last_undegraded":"2024-09-22T01:04:21.652078+0000","last_fullsized":"2024-09-22T01:04:21.652078+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'397","last_scrub_stamp":"2024-09-22T01:04:21.652004+0000","last_deep_scrub":"200'397","last_deep_scrub_stamp":"2024-09-22T01:04:21.652004+0000","last_clean_scrub_stamp":"2024-09-22T01:04:21.652004+0000","objects_scrubbed":9,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:21:33.842697+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0021046329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2222,"num_read_kb":18605,"num_write":395,"num_write_kb":15221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":2886153,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":399,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:28.288063+0000","last_change":"2024-09-22T01:04:28.288121+0000","last_active":"2024-09-22T01:04:28.288063+0000","last_peered":"2024-09-22T01:04:28.288063+0000","last_clean":"2024-09-22T01:04:28.288063+0000","last_became_active":"2024-09-22T01:03:13.744124+0000","last_became_peered":"2024-09-22T01:03:13.744124+0000","last_unstale":"2024-09-22T01:04:28.288063+0000","last_undegraded":"2024-09-22T01:04:28.288063+0000","last_fullsized":"2024-09-22T01:04:28.288063+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:04:28.287843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:04:28.287843+0000","last_clean_scrub_stamp":"2024-09-22T01:04:28.287843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:26:20.429449+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000773721,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"200'398","reported_seq":2416,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:24.495686+0000","last_change":"2024-09-22T01:04:24.495759+0000","last_active":"2024-09-22T01:04:24.495686+0000","last_peered":"2024-09-22T01:04:24.495686+0000","last_clean":"2024-09-22T01:04:24.495686+0000","last_became_active":"2024-09-22T01:01:31.018341+0000","last_became_peered":"2024-09-22T01:01:31.018341+0000","last_unstale":"2024-09-22T01:04:24.495686+0000","last_undegraded":"2024-09-22T01:04:24.495686+0000","last_fullsized":"2024-09-22T01:04:24.495686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'398","last_scrub_stamp":"2024-09-22T01:04:24.495512+0000","last_deep_scrub":"200'398","last_deep_scrub_stamp":"2024-09-22T01:04:24.495512+0000","last_clean_scrub_stamp":"2024-09-22T01:04:24.495512+0000","objects_scrubbed":6,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:49:46.871525+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0037409610000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1662,"num_read_kb":15605,"num_write":377,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1475639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"200'2","reported_seq":340,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:21.006933+0000","last_change":"2024-09-22T01:04:21.006998+0000","last_active":"2024-09-22T01:04:21.006933+0000","last_peered":"2024-09-22T01:04:21.006933+0000","last_clean":"2024-09-22T01:04:21.006933+0000","last_became_active":"2024-09-22T01:01:27.721470+0000","last_became_peered":"2024-09-22T01:01:27.721470+0000","last_unstale":"2024-09-22T01:04:21.006933+0000","last_undegraded":"2024-09-22T01:04:21.006933+0000","last_fullsized":"2024-09-22T01:04:21.006933+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'2","last_scrub_stamp":"2024-09-22T01:04:21.006745+0000","last_deep_scrub":"200'2","last_deep_scrub_stamp":"2024-09-22T01:04:21.006745+0000","last_clean_scrub_stamp":"2024-09-22T01:04:21.006745+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:11:54.262178+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"200'119","reported_seq":538,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:22.292457+0000","last_change":"2024-09-22T01:04:22.292457+0000","last_active":"2024-09-22T01:04:22.292457+0000","last_peered":"2024-09-22T01:04:22.292457+0000","last_clean":"2024-09-22T01:04:22.292457+0000","last_became_active":"2024-09-22T01:00:32.498180+0000","last_became_peered":"2024-09-22T01:00:32.498180+0000","last_unstale":"2024-09-22T01:04:22.292457+0000","last_undegraded":"2024-09-22T01:04:22.292457+0000","last_fullsized":"2024-09-22T01:04:22.292457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'119","last_scrub_stamp":"2024-09-22T01:04:22.292206+0000","last_deep_scrub":"200'119","last_deep_scrub_stamp":"2024-09-22T01:04:22.292206+0000","last_clean_scrub_stamp":"2024-09-22T01:04:22.292206+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:07.331990+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"200'501","reported_seq":10817,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:25.525812+0000","last_change":"2024-09-22T01:04:25.525915+0000","last_active":"2024-09-22T01:04:25.525812+0000","last_peered":"2024-09-22T01:04:25.525812+0000","last_clean":"2024-09-22T01:04:25.525812+0000","last_became_active":"2024-09-22T01:02:39.734805+0000","last_became_peered":"2024-09-22T01:02:39.734805+0000","last_unstale":"2024-09-22T01:04:25.525812+0000","last_undegraded":"2024-09-22T01:04:25.525812+0000","last_fullsized":"2024-09-22T01:04:25.525812+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'501","last_scrub_stamp":"2024-09-22T01:04:25.525598+0000","last_deep_scrub":"200'501","last_deep_scrub_stamp":"2024-09-22T01:04:25.525598+0000","last_clean_scrub_stamp":"2024-09-22T01:04:25.525598+0000","objects_scrubbed":14,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:13:04.881260+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0042366569999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10965,"num_read_kb":27620,"num_write":455,"num_write_kb":13709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":17958941,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"200'612","reported_seq":2608,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:21.696390+0000","last_change":"2024-09-22T01:04:21.696474+0000","last_active":"2024-09-22T01:04:21.696390+0000","last_peered":"2024-09-22T01:04:21.696390+0000","last_clean":"2024-09-22T01:04:21.696390+0000","last_became_active":"2024-09-22T01:03:10.743438+0000","last_became_peered":"2024-09-22T01:03:10.743438+0000","last_unstale":"2024-09-22T01:04:21.696390+0000","last_undegraded":"2024-09-22T01:04:21.696390+0000","last_fullsized":"2024-09-22T01:04:21.696390+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'612","last_scrub_stamp":"2024-09-22T01:04:21.696193+0000","last_deep_scrub":"200'612","last_deep_scrub_stamp":"2024-09-22T01:04:21.696193+0000","last_clean_scrub_stamp":"2024-09-22T01:04:21.696193+0000","objects_scrubbed":9,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:51.195220+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0017953349999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3006,"num_read_kb":7317,"num_write":661,"num_write_kb":8031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":554846,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"200'812","reported_seq":16484,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:24.724893+0000","last_change":"2024-09-22T01:04:24.725094+0000","last_active":"2024-09-22T01:04:24.724893+0000","last_peered":"2024-09-22T01:04:24.724893+0000","last_clean":"2024-09-22T01:04:24.724893+0000","last_became_active":"2024-09-22T01:03:29.784428+0000","last_became_peered":"2024-09-22T01:03:29.784428+0000","last_unstale":"2024-09-22T01:04:24.724893+0000","last_undegraded":"2024-09-22T01:04:24.724893+0000","last_fullsized":"2024-09-22T01:04:24.724893+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'812","last_scrub_stamp":"2024-09-22T01:04:24.724675+0000","last_deep_scrub":"200'812","last_deep_scrub_stamp":"2024-09-22T01:04:24.724675+0000","last_clean_scrub_stamp":"2024-09-22T01:04:24.724675+0000","objects_scrubbed":11,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:12.797663+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0016155010000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":16636,"num_read_kb":40577,"num_write":824,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":13952835,"num_keys_recovered":211,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"200'711","reported_seq":3008,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:20.551063+0000","last_change":"2024-09-22T01:04:20.551141+0000","last_active":"2024-09-22T01:04:20.551063+0000","last_peered":"2024-09-22T01:04:20.551063+0000","last_clean":"2024-09-22T01:04:20.551063+0000","last_became_active":"2024-09-22T01:00:47.716708+0000","last_became_peered":"2024-09-22T01:00:47.716708+0000","last_unstale":"2024-09-22T01:04:20.551063+0000","last_undegraded":"2024-09-22T01:04:20.551063+0000","last_fullsized":"2024-09-22T01:04:20.551063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'711","last_scrub_stamp":"2024-09-22T01:04:20.550889+0000","last_deep_scrub":"200'711","last_deep_scrub_stamp":"2024-09-22T01:04:20.550889+0000","last_clean_scrub_stamp":"2024-09-22T01:04:20.550889+0000","objects_scrubbed":10,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:19.975566+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.010001789000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4517,"num_read_kb":20028,"num_write":818,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"200'425","reported_seq":1607,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:23.056548+0000","last_change":"2024-09-22T01:04:23.056650+0000","last_active":"2024-09-22T01:04:23.056548+0000","last_peered":"2024-09-22T01:04:23.056548+0000","last_clean":"2024-09-22T01:04:23.056548+0000","last_became_active":"2024-09-22T01:01:27.724600+0000","last_became_peered":"2024-09-22T01:01:27.724600+0000","last_unstale":"2024-09-22T01:04:23.056548+0000","last_undegraded":"2024-09-22T01:04:23.056548+0000","last_fullsized":"2024-09-22T01:04:23.056548+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'425","last_scrub_stamp":"2024-09-22T01:04:23.056365+0000","last_deep_scrub":"200'425","last_deep_scrub_stamp":"2024-09-22T01:04:23.056365+0000","last_clean_scrub_stamp":"2024-09-22T01:04:23.056365+0000","objects_scrubbed":7,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:26.565164+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0018099469999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2805,"num_read_kb":21867,"num_write":400,"num_write_kb":11077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2367803,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"200'767","reported_seq":6043,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:25.273520+0000","last_change":"2024-09-22T01:04:25.273611+0000","last_active":"2024-09-22T01:04:25.273520+0000","last_peered":"2024-09-22T01:04:25.273520+0000","last_clean":"2024-09-22T01:04:25.273520+0000","last_became_active":"2024-09-22T01:01:27.732247+0000","last_became_peered":"2024-09-22T01:01:27.732247+0000","last_unstale":"2024-09-22T01:04:25.273520+0000","last_undegraded":"2024-09-22T01:04:25.273520+0000","last_fullsized":"2024-09-22T01:04:25.273520+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'767","last_scrub_stamp":"2024-09-22T01:04:25.273314+0000","last_deep_scrub":"200'767","last_deep_scrub_stamp":"2024-09-22T01:04:25.273314+0000","last_clean_scrub_stamp":"2024-09-22T01:04:25.273314+0000","objects_scrubbed":8,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:45.625801+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001673878,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7435,"num_read_kb":29071,"num_write":891,"num_write_kb":17628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":6690451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"200'495","reported_seq":5238,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:27.294517+0000","last_change":"2024-09-22T01:04:27.294602+0000","last_active":"2024-09-22T01:04:27.294517+0000","last_peered":"2024-09-22T01:04:27.294517+0000","last_clean":"2024-09-22T01:04:27.294517+0000","last_became_active":"2024-09-22T01:03:11.747574+0000","last_became_peered":"2024-09-22T01:03:11.747574+0000","last_unstale":"2024-09-22T01:04:27.294517+0000","last_undegraded":"2024-09-22T01:04:27.294517+0000","last_fullsized":"2024-09-22T01:04:27.294517+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'495","last_scrub_stamp":"2024-09-22T01:04:27.294318+0000","last_deep_scrub":"200'495","last_deep_scrub_stamp":"2024-09-22T01:04:27.294318+0000","last_clean_scrub_stamp":"2024-09-22T01:04:27.294318+0000","objects_scrubbed":9,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:22:58.580452+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0028894910000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3865,"num_read_kb":26291,"num_write":518,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":18082442,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"200'404","reported_seq":5249,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:26.253970+0000","last_change":"2024-09-22T01:04:26.254051+0000","last_active":"2024-09-22T01:04:26.253970+0000","last_peered":"2024-09-22T01:04:26.253970+0000","last_clean":"2024-09-22T01:04:26.253970+0000","last_became_active":"2024-09-22T01:03:09.752301+0000","last_became_peered":"2024-09-22T01:03:09.752301+0000","last_unstale":"2024-09-22T01:04:26.253970+0000","last_undegraded":"2024-09-22T01:04:26.253970+0000","last_fullsized":"2024-09-22T01:04:26.253970+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'404","last_scrub_stamp":"2024-09-22T01:04:26.253787+0000","last_deep_scrub":"200'404","last_deep_scrub_stamp":"2024-09-22T01:04:26.253787+0000","last_clean_scrub_stamp":"2024-09-22T01:04:26.253787+0000","objects_scrubbed":5,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:18:44.810039+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0036899020000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5274,"num_read_kb":10432,"num_write":312,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":1418835,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"200'474","reported_seq":3433,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:22.769995+0000","last_change":"2024-09-22T01:04:22.770083+0000","last_active":"2024-09-22T01:04:22.769995+0000","last_peered":"2024-09-22T01:04:22.769995+0000","last_clean":"2024-09-22T01:04:22.769995+0000","last_became_active":"2024-09-22T01:00:47.722313+0000","last_became_peered":"2024-09-22T01:00:47.722313+0000","last_unstale":"2024-09-22T01:04:22.769995+0000","last_undegraded":"2024-09-22T01:04:22.769995+0000","last_fullsized":"2024-09-22T01:04:22.769995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'474","last_scrub_stamp":"2024-09-22T01:04:22.769777+0000","last_deep_scrub":"200'474","last_deep_scrub_stamp":"2024-09-22T01:04:22.769777+0000","last_clean_scrub_stamp":"2024-09-22T01:04:22.769777+0000","objects_scrubbed":12,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:18:30.410830+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0015314930000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3238,"num_read_kb":14715,"num_write":394,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"200'685","reported_seq":3080,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:29.267737+0000","last_change":"2024-09-22T01:04:29.267833+0000","last_active":"2024-09-22T01:04:29.267737+0000","last_peered":"2024-09-22T01:04:29.267737+0000","last_clean":"2024-09-22T01:04:29.267737+0000","last_became_active":"2024-09-22T01:03:22.751167+0000","last_became_peered":"2024-09-22T01:03:22.751167+0000","last_unstale":"2024-09-22T01:04:29.267737+0000","last_undegraded":"2024-09-22T01:04:29.267737+0000","last_fullsized":"2024-09-22T01:04:29.267737+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'685","last_scrub_stamp":"2024-09-22T01:04:29.267531+0000","last_deep_scrub":"200'685","last_deep_scrub_stamp":"2024-09-22T01:04:29.267531+0000","last_clean_scrub_stamp":"2024-09-22T01:04:29.267531+0000","objects_scrubbed":5,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:26:46.196969+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0014391499999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4549,"num_read_kb":11756,"num_write":798,"num_write_kb":5793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":3407364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"200'455","reported_seq":3290,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:18.560628+0000","last_change":"2024-09-22T01:04:18.560681+0000","last_active":"2024-09-22T01:04:18.560628+0000","last_peered":"2024-09-22T01:04:18.560628+0000","last_clean":"2024-09-22T01:04:18.560628+0000","last_became_active":"2024-09-22T01:00:47.725198+0000","last_became_peered":"2024-09-22T01:00:47.725198+0000","last_unstale":"2024-09-22T01:04:18.560628+0000","last_undegraded":"2024-09-22T01:04:18.560628+0000","last_fullsized":"2024-09-22T01:04:18.560628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'455","last_scrub_stamp":"2024-09-22T01:04:18.560513+0000","last_deep_scrub":"200'455","last_deep_scrub_stamp":"2024-09-22T01:04:18.560513+0000","last_clean_scrub_stamp":"2024-09-22T01:04:18.560513+0000","objects_scrubbed":10,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:59.366056+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0031906880000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3530,"num_read_kb":7508,"num_write":405,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"200'325","reported_seq":6921,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-22T01:04:26.471711+0000","last_change":"2024-09-22T01:04:26.471711+0000","last_active":"2024-09-22T01:04:26.471711+0000","last_peered":"2024-09-22T01:04:26.471711+0000","last_clean":"2024-09-22T01:04:26.471711+0000","last_became_active":"2024-09-22T01:03:10.744943+0000","last_became_peered":"2024-09-22T01:03:10.744943+0000","last_unstale":"2024-09-22T01:04:26.471711+0000","last_undegraded":"2024-09-22T01:04:26.471711+0000","last_fullsized":"2024-09-22T01:04:26.471711+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'325","last_scrub_stamp":"2024-09-22T01:04:26.471440+0000","last_deep_scrub":"200'325","last_deep_scrub_stamp":"2024-09-22T01:04:26.471440+0000","last_clean_scrub_stamp":"2024-09-22T01:04:26.471440+0000","objects_scrubbed":11,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:46.282896+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0027331909999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5696,"num_read_kb":26871,"num_write":338,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":17628733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":256,"num_read":203257,"num_read_kb":760466,"num_write":22625,"num_write_kb":411108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":211294778,"num_keys_recovered":927,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},"log_size":20207,"ondisk_log_size":20207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33536,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338304,"statfs":{"total":96636764160,"available":96602423296,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":52,"seq":223338299432,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":2356,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4530,"internal_metadata":28765774},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542198,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60484,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311356,"statfs":{"total":96636764160,"available":96574828544,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55748,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316092,"statfs":{"total":96636764160,"available":96579678208,"internally_reserved":0,"allocated":2412544,"data_stored":1164479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47424,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324416,"statfs":{"total":96636764160,"available":96588201984,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51584,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320256,"statfs":{"total":96636764160,"available":96583942144,"internally_reserved":0,"allocated":2408448,"data_stored":1164460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542199,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2416640,"data_stored":1164498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":3002368,"data_stored":1754828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:04:42.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-22T01:04:42.842 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-22T01:04:42.842 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-22T01:04:42.842 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T01:04:42.842 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:42.913 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T01:04:42.913 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T01:04:42.914 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:43.018 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T01:04:43.134 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T01:04:43.135 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:43.220 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T01:04:43.220 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T01:04:43.220 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:47.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-22T01:04:49.323 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T01:04:49.323 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-22T01:04:49.323 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:49.421 INFO:tasks.ceph.osd.4:Stopped 2024-09-22T01:04:49.422 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-22T01:04:49.422 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:49.513 INFO:tasks.ceph.osd.5:Stopped 2024-09-22T01:04:49.513 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-22T01:04:49.513 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:53.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-22T01:04:53.530 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-22T01:04:55.615 INFO:tasks.ceph.osd.6:Stopped 2024-09-22T01:04:55.615 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-22T01:04:55.615 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:55.692 INFO:tasks.ceph.osd.7:Stopped 2024-09-22T01:04:55.692 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-22T01:04:55.692 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-22T01:04:55.693 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:55.720 INFO:tasks.ceph.mgr.y:Stopped 2024-09-22T01:04:55.720 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-22T01:04:55.720 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:55.750 INFO:tasks.ceph.mgr.x:Stopped 2024-09-22T01:04:55.750 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-22T01:04:55.751 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-22T01:04:55.751 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:55.776 INFO:tasks.ceph.mon.a:Stopped 2024-09-22T01:04:55.776 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-22T01:04:55.776 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:55.837 INFO:tasks.ceph.mon.c:Stopped 2024-09-22T01:04:55.837 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-22T01:04:55.837 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:04:55.856 INFO:tasks.ceph.mon.b:Stopped 2024-09-22T01:04:55.856 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-22T01:04:55.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-22T01:04:55.899 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T01:04:55.900 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-22T01:04:56.126 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T01:04:56.126 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-22T01:04:56.373 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T01:04:56.373 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-22T01:04:56.538 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T01:04:56.539 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-22T01:04:56.706 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T01:04:56.706 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-22T01:04:56.940 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T01:04:56.940 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-22T01:04:57.147 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T01:04:57.147 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-22T01:04:57.305 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T01:04:57.306 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-22T01:04:57.451 INFO:tasks.ceph:Archiving mon data... 2024-09-22T01:04:57.452 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/data/mon.a.tgz 2024-09-22T01:04:57.452 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-22T01:04:57.458 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.b4L4UDZtCT 2024-09-22T01:04:57.458 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.b4L4UDZtCT 2024-09-22T01:04:57.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.b4L4UDZtCT 2024-09-22T01:04:57.679 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.b4L4UDZtCT is 458KB 2024-09-22T01:04:57.740 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.b4L4UDZtCT 2024-09-22T01:04:57.746 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/data/mon.c.tgz 2024-09-22T01:04:57.746 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-22T01:04:57.795 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.x0sVs1nESA 2024-09-22T01:04:57.795 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.x0sVs1nESA 2024-09-22T01:04:57.996 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.x0sVs1nESA 2024-09-22T01:04:58.058 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.x0sVs1nESA is 626KB 2024-09-22T01:04:58.120 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.x0sVs1nESA 2024-09-22T01:04:58.125 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/data/mon.b.tgz 2024-09-22T01:04:58.125 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-09-22T01:04:58.131 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.nBsHcdPmRF 2024-09-22T01:04:58.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nBsHcdPmRF 2024-09-22T01:04:58.321 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.nBsHcdPmRF 2024-09-22T01:04:58.385 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.nBsHcdPmRF is 626KB 2024-09-22T01:04:58.447 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.nBsHcdPmRF 2024-09-22T01:04:58.452 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-22T01:04:58.452 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T01:04:58.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T01:04:58.515 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T01:04:58.530 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-22T01:04:58.553 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T01:04:58.553 INFO:tasks.ceph:Archiving crash dumps... 2024-09-22T01:04:58.555 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/remote/smithi002/crash 2024-09-22T01:04:58.555 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T01:04:58.571 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/remote/smithi119/crash 2024-09-22T01:04:58.572 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T01:04:58.609 INFO:tasks.ceph:Compressing logs... 2024-09-22T01:04:58.609 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T01:04:58.615 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T01:04:58.635 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34864.log 2024-09-22T01:04:58.636 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34529.log 2024-09-22T01:04:58.636 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40471.log 2024-09-22T01:04:58.636 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34864.log.gz 2024-09-22T01:04:58.638 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34180.log 2024-09-22T01:04:58.638 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34529.log.gz 2024-09-22T01:04:58.638 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40471.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40471.log.gz 2024-09-22T01:04:58.638 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40575.log 2024-09-22T01:04:58.638 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34180.log.gz 2024-09-22T01:04:58.638 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39738.log 2024-09-22T01:04:58.639 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40575.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40575.log.gz -- 2024-09-22T01:04:58.639 INFO:teuthology.orchestra.run.smithi002.stderr: /var/log/ceph/ceph-client.admin.39140.log 2024-09-22T01:04:58.639 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33395.log 2024-09-22T01:04:58.639 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39738.log.gz 2024-09-22T01:04:58.639 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2024-09-22T01:04:58.640 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31959.log 2024-09-22T01:04:58.640 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30763.log 2024-09-22T01:04:58.640 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33395.log: /var/log/ceph/ceph-client.admin.31959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36255.log 2024-09-22T01:04:58.641 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33395.log.gz 2024-09-22T01:04:58.641 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31959.log.gz 2024-09-22T01:04:58.641 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38035.log 2024-09-22T01:04:58.642 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30763.log: /var/log/ceph/ceph-client.admin.36255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36255.log.gz 2024-09-22T01:04:58.642 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30763.log.gz 2024-09-22T01:04:58.642 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37957.log 2024-09-22T01:04:58.642 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-22T01:04:58.643 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38035.log.gz 2024-09-22T01:04:58.643 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32414.log 2024-09-22T01:04:58.643 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37957.log.gz 2024-09-22T01:04:58.643 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41823.log 2024-09-22T01:04:58.644 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32414.log.gz 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35944.log 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41823.log.gz 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32513.log 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33320.log 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35944.log.gz 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32513.log.gz 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39870.log 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32742.log 2024-09-22T01:04:58.645 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33320.log.gz 2024-09-22T01:04:58.646 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.39870.log: -5 --verbose -- /var/log/ceph/ceph-client.0.31588.log 2024-09-22T01:04:58.648 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39870.log.gz 2024-09-22T01:04:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32742.log.gz 2024-09-22T01:04:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40753.log 2024-09-22T01:04:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.0.31588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41174.log 2024-09-22T01:04:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40753.log.gz 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33831.log 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.31588.log.gz 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41174.log: -- /var/log/ceph/ceph-osd.0.log 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41174.log.gz 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33831.log.gz 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31179.log 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31774.log 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31179.log.gz 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41407.log 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31600.log 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31774.log.gz 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41175.log 2024-09-22T01:04:58.649 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41407.log.gz 2024-09-22T01:04:58.653 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40963.log 2024-09-22T01:04:58.653 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31600.log.gz 2024-09-22T01:04:58.653 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40748.log 2024-09-22T01:04:58.653 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41175.log.gz 2024-09-22T01:04:58.653 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37168.log 2024-09-22T01:04:58.653 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40963.log.gz 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41589.log 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40748.log.gz 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41797.log 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37168.log.gz 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33492.log 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41589.log.gz 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37595.log 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41797.log.gz 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31395.log 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33492.log.gz 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40958.log 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37595.log.gz 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31177.log 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31395.log.gz 2024-09-22T01:04:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40627.log 2024-09-22T01:04:58.655 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40958.log.gz 2024-09-22T01:04:58.655 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39169.log 2024-09-22T01:04:58.655 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31177.log.gz 2024-09-22T01:04:58.655 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40627.log.gz 2024-09-22T01:04:58.655 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40180.log 2024-09-22T01:04:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39169.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.39169.log.gz --verbose 2024-09-22T01:04:58.660 INFO:teuthology.orchestra.run.smithi002.stderr: -- /var/log/ceph/ceph-client.admin.32026.log 2024-09-22T01:04:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36193.log 2024-09-22T01:04:58.660 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40180.log.gz 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38193.log 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32026.log.gz 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38864.log 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36193.log.gz 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40967.log 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38193.log.gz 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35986.log 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38864.log.gz 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36484.log 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40967.log.gz 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37818.log 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35986.log.gz 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38515.log 2024-09-22T01:04:58.661 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36484.log.gz 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37818.log.gz 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31161.log 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38515.log.gz 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32224.log 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31161.log.gz 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33290.log 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40028.log 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32224.log.gz 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41355.log 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38954.log 2024-09-22T01:04:58.662 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40028.log.gz 2024-09-22T01:04:58.663 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31601.log 2024-09-22T01:04:58.663 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41355.log.gz 2024-09-22T01:04:58.663 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-22T01:04:58.663 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38954.log.gz 2024-09-22T01:04:58.663 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30772.log 2024-09-22T01:04:58.663 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31601.log.gz 2024-09-22T01:04:58.669 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37172.log 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30970.log 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30772.log.gz 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38478.log 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37172.log.gz 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36966.log 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30970.log.gz 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38478.log.gz 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37240.log 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36966.log.gz 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35835.log 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37240.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.37240.log.gz 2024-09-22T01:04:58.670 INFO:teuthology.orchestra.run.smithi002.stderr: /var/log/ceph/ceph-client.admin.40751.log 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35835.log.gz 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30980.log 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33169.log 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40751.log.gz 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36016.log 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30980.log.gz 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33169.log.gz 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35726.log 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40754.log 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36016.log.gz 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35726.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35726.log.gz 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40754.log.gz 2024-09-22T01:04:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32333.log 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40705.log 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35907.log 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32333.log.gz 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34042.log 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40705.log.gz 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39920.log 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35907.log.gz 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34042.log.gz 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30759.log 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34246.log 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39920.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39920.log.gz 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30759.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.30759.log.gz --verbose 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr: -- /var/log/ceph/ceph-client.admin.38659.log 2024-09-22T01:04:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40601.log 2024-09-22T01:04:58.673 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34246.log.gz 2024-09-22T01:04:58.673 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38609.log 2024-09-22T01:04:58.673 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38659.log.gz 2024-09-22T01:04:58.673 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40601.log.gz 2024-09-22T01:04:58.673 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31189.log 2024-09-22T01:04:58.686 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26496.log 2024-09-22T01:04:58.686 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26544.log 2024-09-22T01:04:58.686 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.26496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26496.log.gz 2024-09-22T01:04:58.686 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26568.log 2024-09-22T01:04:58.686 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.26544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26544.log.gz 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34174.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log/var/log/ceph/ceph-client.admin.26568.log: 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26568.log.gz 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33413.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.34174.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.33413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26472.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26448.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.26472.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26472.log.gz 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.26448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26616.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26448.log.gz 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26520.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.26616.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-22T01:04:58.687 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26616.log.gz 2024-09-22T01:04:58.688 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.26520.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-22T01:04:58.688 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26520.log.gz 2024-09-22T01:04:58.688 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31843.log 2024-09-22T01:04:58.688 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.31843.log: 85.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.34174.log.gz 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26592.log 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- 95.3% /var/log/ceph/ceph-mon.b.log 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/objectstore_tool.33413.log.gz 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.26592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26592.log.gz 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32624.log 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.4.log: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-22T01:04:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.32624.log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.31843.log.gz 2024-09-22T01:04:58.703 INFO:teuthology.orchestra.run.smithi119.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool.32624.log.gz 2024-09-22T01:04:58.705 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32489.log 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38609.log.gz 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31189.log.gz 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40263.log 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34046.log 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32489.log.gz 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30973.log 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40263.log.gz 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34046.log.gz 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32591.log 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37466.log 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30973.log.gz 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 0.0% 2024-09-22T01:04:58.706 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32591.log.gz 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37466.log.gz 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40419.log 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39764.log 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40419.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38329.log 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40419.log.gz 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39764.log.gz 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31180.log 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38119.log 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38329.log.gz 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33675.log 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31180.log.gz 2024-09-22T01:04:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38119.log.gz 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36688.log 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41563.log 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33675.log.gz 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35195.log 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36688.log.gz 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41563.log.gz 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37872.log 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30953.log 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35195.log.gz 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37777.log 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37872.log.gz 2024-09-22T01:04:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30953.log.gz 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42004.log 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5/var/log/ceph/ceph-client.admin.37777.log: --verbose -- /var/log/ceph/ceph-client.admin.39366.log 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37777.log.gz 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42004.log.gz 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34417.log 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39366.log.gz 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38110.log 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31929.log 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34417.log.gz 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40757.log 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38110.log.gz 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31929.log.gz 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36227.log 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26427.log 2024-09-22T01:04:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40757.log.gz 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36281.log 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36227.log.gz 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.26427.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26427.log.gz 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37354.log 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31170.log 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36281.log.gz 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37354.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.37354.log.gz --verbose 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr: -- /var/log/ceph/ceph-client.admin.37431.log 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41875.log 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31170.log.gz 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40154.log 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37431.log.gz 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39037.log 2024-09-22T01:04:58.710 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41875.log.gz 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40154.log.gz 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32631.log 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31868.log 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39037.log.gz 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32376.log 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32631.log.gz 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31868.log.gz 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31473.log 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32932.log 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32376.log.gz 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35544.log 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31473.log.gz 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32932.log.gz 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-22T01:04:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34140.log 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35544.log.gz 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38583.log 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34580.log 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34140.log.gz 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34765.log 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38583.log.gz 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34999.log 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34580.log.gz 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40367.log 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34765.log.gz 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33029.log 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34999.log.gz 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39490.log 2024-09-22T01:04:58.712 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40367.log.gz 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39560.log 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33029.log.gz 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30773.log 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39490.log.gz 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41165.log 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39560.log.gz 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40679.log 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30773.log.gz 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35490.log 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41165.log.gz 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40024.log 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40679.log.gz 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39995.log 2024-09-22T01:04:58.713 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35490.log.gz 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32835.log 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40024.log.gz 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32540.log 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39995.log.gz 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41169.log 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32835.log.gz 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36339.log 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32540.log.gz 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37350.log 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41169.log.gz 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39170.log 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36339.log.gz 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32159.log 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37350.log.gz 2024-09-22T01:04:58.714 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35569.log 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39170.log.gz 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31499.log 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32159.log.gz 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35569.log.gz 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41954.log 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31499.log.gz 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38008.log 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41954.log.gz 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30768.log 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gz 2024-09-22T01:04:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38008.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38008.log.gz 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36726.log 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30768.log.gz 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37473.log 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.36726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40341.log 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36726.log.gz 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33138.log 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37473.log.gz 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40341.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40341.log.gz 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36613.log 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37545.log 2024-09-22T01:04:58.716 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33138.log.gz 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33872.log 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36613.log.gz 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37545.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37545.log.gz 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36072.log 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39659.log 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33872.log.gz 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36072.log.gz 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30622.log 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34583.log 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39659.log.gz 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40445.log 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30622.log.gz 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34583.log.gz 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30771.log 2024-09-22T01:04:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30767.log 2024-09-22T01:04:58.718 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40445.log.gz 2024-09-22T01:04:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30719.log 2024-09-22T01:04:58.718 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30771.log.gz 2024-09-22T01:04:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26539.log 2024-09-22T01:04:58.718 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30767.log.gz 2024-09-22T01:04:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30719.log.gz 2024-09-22T01:04:58.731 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35659.log 2024-09-22T01:04:58.731 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.26539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34207.log 2024-09-22T01:04:58.731 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26539.log.gz 2024-09-22T01:04:58.731 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-09-22T01:04:58.731 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35659.log.gz 2024-09-22T01:04:58.731 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34207.log.gz 2024-09-22T01:04:58.731 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34388.log 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32708.log 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35655.log 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34388.log.gz 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31181.log 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32708.log.gz 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35126.log 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35655.log.gz 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40961.log 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31181.log: /var/log/ceph/ceph-client.admin.35126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36369.log 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31181.log.gz 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35126.log.gz 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40961.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40961.log.gz 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37984.log 2024-09-22T01:04:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36563.log 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36369.log.gz 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31599.log 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37984.log.gz 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36563.log.gz 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34918.log 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41147.log 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31599.log.gz 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35102.log 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34918.log.gz 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41147.log.gz 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38375.log 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30968.log 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35102.log.gz 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38375.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.38375.log.gz 2024-09-22T01:04:58.733 INFO:teuthology.orchestra.run.smithi002.stderr: /var/log/ceph/ceph-client.admin.32787.log 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32658.log 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30968.log.gz 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41849.log 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32787.log.gz 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32658.log.gz 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39400.log 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41849.log.gz 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38791.log 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39400.log.gz 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32071.log 2024-09-22T01:04:58.734 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38791.log.gz 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35456.log 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32263.log 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32071.log.gz 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39825.log 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35456.log.gz 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32263.log.gz 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40966.log 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34826.log 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39825.log.gz 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31714.log 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40966.log.gz 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34826.log.gz 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39305.log 2024-09-22T01:04:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35752.log 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31714.log.gz 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41667.log 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39305.log.gz 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35752.log.gz 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37620.log 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41667.log.gz 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35407.log 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.37620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37620.log.gz 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36855.log 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40653.log 2024-09-22T01:04:58.736 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35407.log.gz 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32982.log 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36855.log.gz 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41693.log 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40653.log.gz 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38288.log 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32982.log.gz 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36965.log 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41693.log.gz/var/log/ceph/ceph-client.admin.38288.log: 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38288.log.gz -5 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36012.log 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36965.log.gz 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33236.log 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33428.log 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr: 28.4% -- replaced with /var/log/ceph/ceph-client.admin.36012.log.gz 2024-09-22T01:04:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33236.log.gz 2024-09-22T01:04:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33583.log 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37089.log 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33428.log.gz/var/log/ceph/ceph-client.admin.33583.log: 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36480.log 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33583.log.gz 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34860.log 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37089.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37089.log.gz 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30647.log 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36480.log.gz 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33609.log 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34860.log: /var/log/ceph/ceph-client.admin.30647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34860.log.gz 2024-09-22T01:04:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30597.log 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30647.log.gz 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34073.log 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33609.log: /var/log/ceph/ceph-client.admin.30597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38825.log 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33609.log.gz 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30597.log.gz 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34073.log.gz 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30961.log 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36920.log 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38825.log.gz 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30969.log 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30961.log.gz 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36920.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36920.log.gz 2024-09-22T01:04:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30969.log.gz 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40549.log 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40549.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35776.log 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40549.log.gz 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36647.log 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34944.log 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35776.log.gz 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36647.log.gz 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40289.log 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41745.log 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34944.log.gz 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40289.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31421.log 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41745.log.gz 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37281.log 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40946.log 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31421.log.gz 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37281.log.gz 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41172.log 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30667.log 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40946.log.gz 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41172.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33700.log 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30667.log.gz 2024-09-22T01:04:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-09-22T01:04:58.751 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30693.log 2024-09-22T01:04:58.751 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33700.log.gz 2024-09-22T01:04:58.751 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-09-22T01:04:58.751 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41615.log 2024-09-22T01:04:58.751 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30693.log.gz 2024-09-22T01:04:58.751 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-09-22T01:04:58.755 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38750.log 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41615.log.gz 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38557.log 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38750.log.gz 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35160.log 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38557.log: /var/log/ceph/ceph-client.admin.40758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38557.log.gz 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36779.log 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30770.log 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35160.log: /var/log/ceph/ceph-client.admin.36779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35160.log.gz 2024-09-22T01:04:58.756 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36779.log.gz 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41641.log 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32544.log 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30770.log: /var/log/ceph/ceph-client.admin.41641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30770.log.gz 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41641.log.gz 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38983.log 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40497.log 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32544.log.gz 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38983.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38635.log 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40497.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40497.log.gz 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38635.log.gz 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39946.log 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32684.log 2024-09-22T01:04:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39946.log.gz 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41485.log 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32684.log.gz 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37682.log 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41485.log.gz 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40755.log 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37682.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37682.log.gz 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39453.log 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40755.log.gz 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38401.log 2024-09-22T01:04:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32873.log 2024-09-22T01:04:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39453.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39453.log.gz 2024-09-22T01:04:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38401.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39223.log 2024-09-22T01:04:58.762 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32873.log.gz 2024-09-22T01:04:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35033.log 2024-09-22T01:04:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35362.log 2024-09-22T01:04:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39223.log.gz 2024-09-22T01:04:58.763 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35033.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33996.log 2024-09-22T01:04:58.763 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.764 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35362.log.gz 2024-09-22T01:04:58.764 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40076.log 2024-09-22T01:04:58.764 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33487.log 2024-09-22T01:04:58.765 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33996.log.gz 2024-09-22T01:04:58.765 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40076.log.gz 2024-09-22T01:04:58.765 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39972.log 2024-09-22T01:04:58.765 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33487.log.gz 2024-09-22T01:04:58.765 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32131.log 2024-09-22T01:04:58.766 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37933.log 2024-09-22T01:04:58.766 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39972.log.gz 2024-09-22T01:04:58.766 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.32131.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36068.log 2024-09-22T01:04:58.766 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32131.log.gz 2024-09-22T01:04:58.767 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37933.log.gz 2024-09-22T01:04:58.767 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35221.log 2024-09-22T01:04:58.767 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31815.log 2024-09-22T01:04:58.768 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36068.log.gz 2024-09-22T01:04:58.768 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35221.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33066.log 2024-09-22T01:04:58.769 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.769 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31815.log.gz 2024-09-22T01:04:58.769 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40102.log 2024-09-22T01:04:58.769 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2024-09-22T01:04:58.770 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33066.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33066.log.gz 2024-09-22T01:04:58.770 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41719.log 2024-09-22T01:04:58.770 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40102.log.gz 2024-09-22T01:04:58.771 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41901.log: 27.8% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz 2024-09-22T01:04:58.771 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40759.log 2024-09-22T01:04:58.771 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31369.log 2024-09-22T01:04:58.772 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41719.log: /var/log/ceph/ceph-client.admin.40759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41719.log.gz 2024-09-22T01:04:58.772 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40759.log.gz 2024-09-22T01:04:58.772 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31188.log 2024-09-22T01:04:58.772 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33873.log 2024-09-22T01:04:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31369.log.gz 2024-09-22T01:04:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31188.log.gz 2024-09-22T01:04:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33919.log 2024-09-22T01:04:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33873.log.gz 2024-09-22T01:04:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39655.log 2024-09-22T01:04:58.775 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31555.log 2024-09-22T01:04:58.775 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33919.log.gz 2024-09-22T01:04:58.775 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39655.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39655.log.gz 2024-09-22T01:04:58.775 INFO:teuthology.orchestra.run.smithi002.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39556.log 2024-09-22T01:04:58.776 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31555.log.gz 2024-09-22T01:04:58.776 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31525.log 2024-09-22T01:04:58.776 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40523.log 2024-09-22T01:04:58.777 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.39556.log: /var/log/ceph/ceph-client.admin.31525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39556.log.gz 2024-09-22T01:04:58.777 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31525.log.gz 2024-09-22T01:04:58.777 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41433.log 2024-09-22T01:04:58.778 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40206.log 2024-09-22T01:04:58.779 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40523.log.gz 2024-09-22T01:04:58.779 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41433.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41433.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33777.log 2024-09-22T01:04:58.779 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.779 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40206.log.gz 2024-09-22T01:04:58.779 INFO:teuthology.orchestra.run.smithi119.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-22T01:04:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26280.log 2024-09-22T01:04:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34727.log 2024-09-22T01:04:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33777.log.gz 2024-09-22T01:04:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.26280.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26280.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30522.log 2024-09-22T01:04:58.781 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34727.log.gz 2024-09-22T01:04:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36439.log 2024-09-22T01:04:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41511.log 2024-09-22T01:04:58.782 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.30522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30522.log.gz 2024-09-22T01:04:58.782 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36439.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40964.log 2024-09-22T01:04:58.782 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.782 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41511.log.gz 2024-09-22T01:04:58.783 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38238.log 2024-09-22T01:04:58.783 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37118.log 2024-09-22T01:04:58.783 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40964.log.gz 2024-09-22T01:04:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.38238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38238.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21426.log 2024-09-22T01:04:58.784 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37118.log.gz 2024-09-22T01:04:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41980.log 2024-09-22T01:04:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40315.log 2024-09-22T01:04:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph.tmp-client.admin.21426.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21426.log.gz 2024-09-22T01:04:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41980.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40232.log 2024-09-22T01:04:58.786 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40315.log.gz 2024-09-22T01:04:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41381.log 2024-09-22T01:04:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33633.log 2024-09-22T01:04:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40232.log.gz 2024-09-22T01:04:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41381.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35271.log 2024-09-22T01:04:58.788 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33633.log.gz 2024-09-22T01:04:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41166.log 2024-09-22T01:04:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41771.log 2024-09-22T01:04:58.789 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35271.log.gz 2024-09-22T01:04:58.789 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41166.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.41166.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.34322.log 2024-09-22T01:04:58.789 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31447.log/var/log/ceph/ceph-client.admin.41771.log: 2024-09-22T01:04:58.790 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41771.log.gz 2024-09-22T01:04:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34322.log.gz 2024-09-22T01:04:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41927.log 2024-09-22T01:04:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.31447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37287.log 2024-09-22T01:04:58.792 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31447.log.gz 2024-09-22T01:04:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41927.log.gz 2024-09-22T01:04:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30981.log 2024-09-22T01:04:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40128.log 2024-09-22T01:04:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.37287.log: /var/log/ceph/ceph-client.admin.30981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37287.log.gz 2024-09-22T01:04:58.793 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30981.log.gz 2024-09-22T01:04:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34492.log 2024-09-22T01:04:58.794 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33952.log 2024-09-22T01:04:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40128.log.gz 2024-09-22T01:04:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.34492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34492.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41170.log 2024-09-22T01:04:58.795 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33952.log.gz 2024-09-22T01:04:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33743.log 2024-09-22T01:04:58.796 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36829.log 2024-09-22T01:04:58.796 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41170.log.gz 2024-09-22T01:04:58.796 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.33743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33743.log.gz 2024-09-22T01:04:58.796 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34363.log 2024-09-22T01:04:58.797 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36829.log.gz 2024-09-22T01:04:58.797 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35831.log 2024-09-22T01:04:58.798 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-09-22T01:04:58.798 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35831.log: /var/log/ceph/ceph-client.admin.34363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34363.log.gz 2024-09-22T01:04:58.798 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35831.log.gz 2024-09-22T01:04:58.798 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41537.log 2024-09-22T01:04:58.799 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40393.log 2024-09-22T01:04:58.800 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-09-22T01:04:58.800 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41537.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35320.log 2024-09-22T01:04:58.800 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:04:58.800 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40393.log.gz 2024-09-22T01:04:58.801 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.35320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35320.log.gz 2024-09-22T01:04:58.921 INFO:teuthology.orchestra.run.smithi119.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-22T01:04:58.937 INFO:teuthology.orchestra.run.smithi002.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-22T01:04:59.115 INFO:teuthology.orchestra.run.smithi002.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-22T01:05:00.156 INFO:teuthology.orchestra.run.smithi002.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-22T01:05:00.318 INFO:teuthology.orchestra.run.smithi119.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-22T01:05:00.807 INFO:teuthology.orchestra.run.smithi119.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-22T01:05:00.848 INFO:teuthology.orchestra.run.smithi119.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-22T01:05:00.904 INFO:teuthology.orchestra.run.smithi002.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-22T01:05:01.871 INFO:teuthology.orchestra.run.smithi002.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-22T01:05:02.489 INFO:teuthology.orchestra.run.smithi119.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-22T01:05:02.490 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-22T01:05:02.490 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m3.832s 2024-09-22T01:05:02.490 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m9.723s 2024-09-22T01:05:02.490 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m0.398s 2024-09-22T01:05:03.167 INFO:teuthology.orchestra.run.smithi002.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-22T01:05:03.168 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-22T01:05:03.169 INFO:teuthology.orchestra.run.smithi002.stderr:real 0m4.551s 2024-09-22T01:05:03.169 INFO:teuthology.orchestra.run.smithi002.stderr:user 0m11.851s 2024-09-22T01:05:03.169 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m0.552s 2024-09-22T01:05:03.170 INFO:tasks.ceph:Archiving logs... 2024-09-22T01:05:03.170 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/remote/smithi002/log 2024-09-22T01:05:03.171 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T01:05:04.616 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/remote/smithi119/log 2024-09-22T01:05:04.616 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T01:05:05.635 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-22T01:05:05.645 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-22T01:05:05.645 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T01:05:05.647 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T01:05:05.731 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-22T01:05:05.740 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-22T01:05:05.741 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T01:05:05.743 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T01:05:05.754 INFO:teuthology.orchestra.run.smithi002.stdout: remote refid st t when poll reach delay offset jitter 2024-09-22T01:05:05.754 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================== 2024-09-22T01:05:05.754 INFO:teuthology.orchestra.run.smithi002.stdout:*hv01.front.sepi 67.205.162.81 3 u 37 64 177 0.088 -1.217 0.873 2024-09-22T01:05:05.754 INFO:teuthology.orchestra.run.smithi002.stdout:+hv02.front.sepi 63.231.80.2 3 u 45 64 177 0.096 +0.066 0.791 2024-09-22T01:05:05.754 INFO:teuthology.orchestra.run.smithi002.stdout:+hv03.front.sepi 74.6.168.72 3 u 37 64 177 0.104 -1.691 0.743 2024-09-22T01:05:05.754 INFO:teuthology.orchestra.run.smithi002.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:05:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: remote refid st t when poll reach delay offset jitter 2024-09-22T01:05:05.756 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================== 2024-09-22T01:05:05.756 INFO:teuthology.orchestra.run.smithi119.stdout:*hv01.front.sepi 67.205.162.81 3 u 42 64 177 0.094 -1.329 0.960 2024-09-22T01:05:05.756 INFO:teuthology.orchestra.run.smithi119.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 64 177 0.084 -0.050 1.002 2024-09-22T01:05:05.756 INFO:teuthology.orchestra.run.smithi119.stdout:+hv03.front.sepi 74.6.168.72 3 u 44 64 177 0.133 -1.850 0.934 2024-09-22T01:05:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:05:05.757 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-22T01:05:05.766 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-22T01:05:05.766 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-22T01:05:05.775 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-22T01:05:05.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-22T01:05:05.792 INFO:teuthology.task.internal:Duration was 900.761766 seconds 2024-09-22T01:05:05.792 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-22T01:05:05.801 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-22T01:05:05.801 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T01:05:05.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T01:05:05.840 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-22T01:05:05.840 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2024-09-22T01:05:05.840 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T01:05:05.896 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-09-22T01:05:05.896 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T01:05:05.906 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-22T01:05:05.906 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T01:05:05.939 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T01:05:05.977 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-22T01:05:05.977 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T01:05:06.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T01:05:06.089 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-22T01:05:06.105 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-22T01:05:06.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T01:05:06.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T01:05:06.131 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-22T01:05:06.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T01:05:06.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T01:05:06.172 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2024-09-22T01:05:06.183 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-09-22T01:05:06.299 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T01:05:06.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:05:06.304 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T01:05:06.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:05:06.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-22T01:05:06.355 INFO:teuthology.task.internal:Transferring archived files... 2024-09-22T01:05:06.355 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/remote/smithi002 2024-09-22T01:05:06.355 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T01:05:06.382 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914593/remote/smithi119 2024-09-22T01:05:06.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T01:05:06.408 INFO:teuthology.task.internal:Removing archive directory... 2024-09-22T01:05:06.409 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T01:05:06.419 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T01:05:06.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-22T01:05:06.464 INFO:teuthology.task.internal:Not uploading archives. 2024-09-22T01:05:06.464 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-22T01:05:06.473 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-22T01:05:06.473 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T01:05:06.476 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T01:05:06.479 INFO:teuthology.orchestra.run.smithi002.stdout: 658644 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 01:05 /home/ubuntu/cephtest 2024-09-22T01:05:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: 658465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 01:05 /home/ubuntu/cephtest 2024-09-22T01:05:06.502 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-22T01:05:06.513 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-22T01:05:06.535 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 900.7617661952972 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-22T01:05:06.536 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T01:05:06.623 INFO:teuthology.run:pass